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I. INTRODUCTION 


A. PURPOSE OF THE REPORT 

Over the past several years, a number of new programs, policies, and initiatives designed to 
promote the employment of people with disabilities have been implemented by state and federal 
agencies, and are being evaluated to greater and lesser degrees. The purpose of this report is to 
provide a review of the recent evaluation activities being conducted for these new initiatives, as 
well as some existing programs that serve people with disabilities. The review is intended to 
provide policymakers, researchers, and others interested in efforts designed to promote the 
employment of people with disabilities a single source for information on the nature of the 
initiatives and the evaluation efforts that have been recently completed or are currently under 
way and the findings to date related to the effectiveness of these initiatives. This broad review is 
also intended to provide some evidence of the progress we are making. The report also suggests 
avenues where further efforts and progress might be warranted. 

Our review focuses on studies and initiatives that — 


• Represent a federally sponsored program, policy, or initiative designed specifically to 
improve employment of the working-age adult population with disabilities, and 

• Evaluate the effectiveness of the initiative or program (the evaluation must have been 
completed by 2000, or be pending implementation or completion). 


Using the above criteria, we identified 27 initiatives or programs and their associated 
evaluations for review. These are briefly described in Exhibit 1. Because of resource constraints, 
we did not review initiatives designed to improve the adult employment outcomes of youth with 
disabilities, such as the Social Security Administration (SSA) sponsored Youth Transition 
Demonstrations. We also did not review small-scale studies evaluating the effectiveness of 
specific clinical, supported employment, or vocational rehabilitation (VR) approaches. We only 
looked at information related to the major federal programs serving people with disabilities, 
general legislation and policies, and initiatives that were fairly large-scale in nature. 

To provide context for the discussion of findings, in the remainder of this introductory 
section we describe the general theory of the labor market and factors believed to affect whether 
an individual with a disability becomes and remains employed. The findings from our review are 
presented in Section II. In the final section, we provide summary descriptions of each of the 27 
initiatives reviewed. 
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EXHIBIT 1 


EMPLOYMENT-RELATED INITIATIVES REVIEWED 


Program/Policy/Initiative 

Oversight/Sponsoring Agency 

Year(s) Implemented 

Target Population 

Employment Barriers Addressed 

Legislation 

Americans with Disabilities 
Act* 

Congress/EEOC 

1999 

People with 
disabilities 

Workplace discrimination 

Lack of workplace accommodations 

Balanced Budget Act* 

Congress/CMS 

1997 

Working people 
with disabilities 

Lack of adequate health supports 
Loss of public health insurance as 
earnings rise 

Ticket to Work and Work 
Incentives Improvement Act* 

Congress/S S A/CMS 

1999 

People with 
disabilities, with 
particular emphasis 
on SSI/ DI 
beneficiaries 

Lack of adequate health supports 
Loss of public health insurance as 
earnings rise 

Risk of cash benefit loss due to 
employment 

Lack of employment-related 
supports 

Lack of information about SSA 
work incentives 

Workforce Investment Act* 

Congress/DOL/RSA 

1998 

Adults, dislocated 
workers, and youth 

System fragmentation 

Lack of consumer control in career 
development 

Centers for Medicare and Medicaid Services (CMS) 

Demonstrations to Maintain 
Independence and Employment 

CMS 

Various years since 
2000 

Working adults with 
disabilities or 
potentially disabling 
conditions 

Lack of health insurance/adequate 
health supports to prevent 
conditions from becoming disabling 

Incentives to discontinue work to 
qualify for public health insurance 

Medicaid Buy-In Programs 

CMS 

Various years since 
1992 

Working adults with 
disabilities 

Lack of adequate health supports 
Loss of public health insurance as 


earnings rise 





EXHIBIT 1 ( continued ) 


Program/Policy/Initiative 

Oversight/Sponsoring Agency Year(s) Implemented 

Target Population 

Employment Barriers Addressed 

Medicaid Infrastructure Grants 

CMS 

Various years since 
2001 

Adults and 
transition-age youth 
with disabilities 

V ariety of barriers addressed, 
depending on the focus of 
individual state efforts 

Department of Education (ED) 

Federal/State Vocational 
Rehabilitation ( VR) Services 
Program 

RSA 

1973 

People with 
disabilities who 
want to work 

Lack of employment-related 
supports 

Lack of skills/training 

Projects with Industry 

RSA 

1968 

People with 
disabilities who 
want to work 

Lack of skills/training 
Lack of employment-related 
supports 

Lack of business involvement in 
employment programs 

Systems Change/State 
Partnership Initiative 

RSA/SSA 

1998 

SSI/DI beneficiaries 
& others with 
disabilities who 
want to work 

System complexity and lack of 
coordination 

Department of Labor (DOL) 

Customized Employment 
Grants 

DOL 

2001-2003 

People with 
disabilities who 
want to work 

Lack of flexible, individualized 
employment supports 

Disability Employment Grants 

DOL 

1998 and 2002 

People with 
disabilities who 
want to work 

Lack of employment opportunities 
above entry-level positions 

Disability Program Navigator 

DOL/SSA/state One-stops Various years since 

2003 

One-stop users with 
disabilities 

System complexity and lack of 
accessibility of the One-stop system 
to people with disabilities 

Work Incentive Grants 

DOL 

2000-2004 

One-stop users with 
disabilities 

Lack of accessibility of the One- 
stop system to people with 
disabilities 





EXHIBIT 1 ( continued ) 


Program/Policy/Initiative 

Oversight/Sponsoring Agency 

Year(s) Implemented 

Target Population 

Employment Barriers Addressed 

Internal Revenue Service (IRS) 

Barrier Removal Tax Deduction 

IRS 


1976 (amended in 
1990) 

Businesses of any 
size 

Costs to employers to 
hire/retain/accommodate workers 
with disabilities 

Disabled Access Tax Credit 

IRS 



Small businesses 

Costs to employers to 
hire/retain/accommodate workers 
with disabilities 

Work Opportunity Tax Credit 

IRS 


1996 

Businesses of any 
size 

Costs to employers to 
hire/retain/accommodate workers 
with disabilities 

Department of Veterans Affairs (VA) 

Vocational Rehabilitation and 
Employment Program 

VA 


1918 (in various 
forms) 

Veterans with 

service-connected 

disabilities 

Lack of employment -related 
supports 

Lack of skills/training 

Social Security Administration (SSA) 

Accelerated Benefits 
Demonstration 

SSA 


2007 

Uninsured DI 
beneficiaries in the 
Medicare 24-month 
waiting period 

Lack of health insurance/adequate 
health supports during Medicare 24- 
month waiting period 

Benefit Offset National 
Demonstration 

SSA 


Pending 

DI beneficiaries 

Loss of DI benefits as earnings rise 

Benefits Planning, Assistance 
and Outreach/ Work Incentives 
Planning and Assistance 

SSA 


2001 

SSI and DI 
beneficiaries 

Lack of information about SSA 
work incentives 

Employment Support 
Representatives/ Area Work 
Incentive Coordinators 

SSA 


2000 

SSI and DI 
beneficiaries 

Lack of information about SSA 
work incentives 

Florida Freedom Initiative 

SSA/ CMS/HHS/Florida 
Agency for Persons with 
Disabilities 

2005 

Adult SSI recipients 
with developmental 
disabilities 

Loss of SSI and Medicaid as 
earnings and assets increase 

Lack of employment -related 
supports 







EXHIBIT 1 ( continued ) 


Program/Policy/Initiative 

Oversight/Sponsoring Agency 

Year(s) Implemented 

Target Population 

Employment Barriers Addressed 

Mental Health Treatment Study 

SSA 

2006 

DI beneficiaries 
with schizophrenia 
or affective disorder 

Lack of adequate medical, 
behavioral, and employment-related 
supports and 

coordination/integration among 
these supports 

Substantial Gainful Activity 
Level Increase 

SSA 

1999 

SSI and DI 
beneficiaries 

Loss of DI benefits as earnings rise 

Ticket to Work Program 

SSA 

2002 

SSI and DI 
beneficiaries 

Lack of employment-related 
supports 

Substance Abuse and Mental Health Services Administration (SAMHSA) 

Employment Intervention 
Demonstration Program 

SAMHSA 

1995-2000 

Adults with serious 
mental illness 

Lack of adequate medical, 
behavioral, and employment-related 
supports and 

coordination/integration among 
these supports 


* The information shown in the exhibit refers exclusively to the provisions relevant to the employment of people with disabilities. 





B. FACTORS AFFECTING THE EMPLOYMENT OF PEOPLE WITH DISABILITIES 


Recent efforts designed to promote the employment of people with disabilities and attempts 
to evaluate their effectiveness cannot be assessed without an understanding of the reasons why 
employment can be beneficial and the factors likely to affect the employment of people with 
disabilities. In this section, we present a brief discussion of these issues to provide a framework 
and context for the discussion of the findings. 


1. Why the Employment of People with Disabilities Matters 

The employment rates of working-age people with disabilities are low relative to their 
counterparts without disabilities, and there is evidence that the rates have been declining since 
the 1990s (Stapleton, Burkhauser and Houtenville 2004). In 2005, the employment rate of people 
with disabilities was just 38 percent, compared with 78 percent among people without disabilities 
(Rehabilitation Research and Training Center on Disability Demographics and Statistics 2005). 

A very low employment rate for any working-age group in our society is cause for concern 
for several reasons: 


• From the individual perspective, market work is the principal source of income in all 
modem societies and is key to financial independence and well-being. Employment 
and contributing to family and societal productivity is also an important component of 
self-esteem. 

• From the employer perspective, if individuals are unable or unwilling to easily 
participate in the labor market, it reduces the pool of qualified. This makes it more 
difficult and costly for employers to access the labor they need to produce their goods 
and services. 

• From the government and taxpayer perspective, low employment rates are 
undesirable because local, state, and federal governments (supported by taxpayers) 
benefit from the taxes levied on earnings and greater consumption in response to 
higher income. Governments also benefit from high employment rates because people 
reduce their dependence on public support programs that are financed by government 
expenditures. 

• From society’s perspective, low employment means that some human capital is not 
being put to productive use, so aggregate productivity and welfare might not be 
maximized. When the skills and talents of individuals are not put to productive uses, 
the lost productivity cannot be recaptured. 


Government investments in programs, policies, and initiatives designed to increase the 
employment of working-age people with disabilities are desirable because they can lead to 
greater individual well-being, increased government revenues, reduced government spending, 
and net gains to society. 
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2. An Overview of the Labor Market and Factors Affecting the Employment of People 

with Disabilities 

The labor market is comprised of two primary players: individuals who supply labor and 
firms that demand labor. The interactions of these players, along with many different factors that 
influence their individual decisions regarding employment, will affect the employment of people 
with disabilities in the aggregate. Below, we briefly discuss the factors that can affect the 
willingness of individuals with disabilities to supply labor; the likelihood that employers will 
hire and retain employees with disabilities; and the efficiency with which individuals are 
matched to jobs in the labor market. 


a. Individuals 

Economic theory posits that the amount of labor individuals are willing to supply (that is, 
the number of hours individuals are willing to work) will depend on their preferences and an 
hours/eamings tradeoff. Individuals must choose how to allocate their limited time between 
market work activities and all other activities. Economists refer to non-work activities as 
“leisure,” but these include all forms of unpaid work, household work, dependent care, and self- 
care. Market work is necessary to obtain earnings, which are used to purchase goods and 
services. Therefore, the allocation of hours to market work and other activities represents a 
choice based on the individual’s preferences and the tradeoff between the consumption of goods 
and services and the consumption of leisure. Many factors will affect the perceived net benefits 
to working by an individual, and will determine whether an individual decides to become 
employed and the number of hours he or she will decide to work. 

Compensation and Benefits. How much an individual will earn from work, along with 
other benefits (for example, health insurance, retirement benefits, paid leave) will certainly 
influence his or her decision to engage in work. All else constant, the higher the wages and 
benefits, the more likely an individual will be willing to supply labor. Many factors will 
influence wages and benefits associated with specific jobs and occupations, and for specific 
individuals within a job or occupation. In general, anything that affects productivity (or 
perceived productivity) will affect the wage rate. The more productive an individual is (as 
measured by the market value of the output produced), the higher the wage the individual will 
command in the labor market. Factors that tend to affect wage rates include education, skills, 
training, and experience. If a disability reduces productivity by eroding skills or abilities, it can 
also reduce the wage rate offered to the individual. At lower wage rates, individuals may be 
unwilling to work or to work as many hours. 

Availability of Non-Labor Income and Resources. If an individual has income available 
from sources other than earnings, it can diminish the need and desire to engage in employment. 
Other sources of income might include spousal earnings, savings, private disability benefits, and 
public health insurance and income assistance, such as Social Security Disability Insurance (DI), 
Supplemental Security Income (SSI), Medicare, Medicaid, food stamps, and housing assistance. 
The more income and resources available to the individual from other sources, the lower the 
value of work. 
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Preferences for Work and Non-Work Activities. An individual’s own preferences for 
spending time working versus non-work activities will influence the likelihood of working and 
the hours of labor supplied. Time is a limited resource. How one allocates his or her limited time 
will be based on individual preferences for engaging in paid work (for purposes of earning 
income that will enable the purchase of goods and services) and non- work activities. Family 
obligations, time required for health or personal care activities, how one derives enjoyment and 
self-worth, and expected remaining years of life will affect the value of work relative to other 
activities. 

Resources/Investments Required to Engage in Work. Work-related expenses or 
investments may be required to successfully engage in paid work; for example, transportation to 
and from work, business attire or uniforms, or specific kinds of tools, equipment, or supplies. All 
else equal, the higher the cost of work-related items borne by the individual, the lower the net 
value of work from the individual’s perspective. 

Information. Information plays an important role in the individual’s decision-making 
process. If individuals lack adequate, or accurate information about the process and resources 
available for finding and maintaining employment, the impact of earnings on benefits and 
income, or other consequences of working or not working, they might make decisions about 
work that are less than optimal for them. Inaccurate information can lead to the individual to 
work too little or too much, relative to the optimal amount. For instance, if the individual thinks 
that earnings will have a more negative impact on public benefits than they actually do, he or she 
might work too little, but the opposite might be true if the individual fails to anticipate the 
negative effect of earnings on benefits. 


b. Firms 

Economic theory predicts that the amount of labor firms demand will depend on the demand 
for the firm’s product, the productivity of labor (and other inputs), the wage rate and other costs 
firms must incur to hire labor, and the costs associated with other inputs that are substitutes or 
complements to labor. The theoretical profit-maximizing firm will demand the amount of labor 
at which the value of the additional output produced by the last worker hired is just equal to the 
cost of hiring that worker. Therefore, anything that affects the cost or productivity of labor will 
affect the amount of labor demanded by a firm. Factors that can affect the cost or productivity of 
labor include: 

Human Capital. Human capital refers to the stock of skills and abilities embodied in labor. 
Greater skills, knowledge, and experience can lead to greater productivity. Productivity, or the 
value of what is being produced by labor, is taken in consideration relative to the cost of labor in 
a firm’s hiring decisions. Demand for labor will be higher when productivity is high relative to 
the costs. Factors that might erode productivity, such as poor health or changes in technology 
that make existing skills obsolete, will reduce the demand for labor. 

Compensation and Benefits. How much the firm must pay labor in the form of wages and 
benefits will affect the firm’s demand for labor. As noted previously, wages are usually higher 
the greater the value of the labor output. Factors other than productivity can affect wages and 
benefits, however, such as: the availability of labor and competition in the local market; state and 
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federal regulations, such as minimum wage and workers’ compensation laws; the influence of 
labor unions; and wage discrimination by the employer. 

Accommodations. The accommodations that employers might need to make for workers 
with disabilities represent a cost of employment. Accommodations might include the purchase of 
special equipment (such as modified computers for persons with impaired eyesight), providing 
wheelchair access, flexible hours to accommodate shift work, and allowing employees to work 
from home. Non-disability related accommodations might also be required by certain 
individuals; for example, child care, or flexible schedules to accommodate employees with 
children or long commutes. Economic theory suggests that high accommodation costs would 
likely result in a reduced demand for labor among individuals requiring high-cost 
accommodations. In an attempt to avoid having to make such accommodations, employers might 
avoid hiring such individuals, or might shift the cost of those accommodations to the employees 
who require accommodation in the form of lower wages. 1 

Taxes, Subsidies, and Government Regulations. There are several provisions in the U.S. 
tax code designed to reduce the employer tax burden and offset the potential or realized costs of 
hiring individuals with disabilities, thereby increasing a firm’s incentive to employ people with 
disabilities. These take the form of wage subsidies, tax credits, and tax deductions. A number of 
other, more general government regulations, such as occupational health and safety standards, 
family leave requirements, and the FICA tax, serve to increase the cost of labor to the firm. 


c. Job-Employee Matching 

Economic theory provides a framework for analyzing job-search behavior. Individuals vary 
in their abilities, education, work experiences, and characteristics, and jobs vary in their skill and 
knowledge requirements. Economists often describe the labor market as a matching process in 
which workers with varying characteristics are matched with appropriate jobs. The economic 
framework for this process is useful for understanding how environmental factors affect an 
individual’s job search and ultimately, employment. 

According to this theory, individuals entering the job market are constrained by their 
available resources to accept an offer of employment within a specified timeframe. Initial assets 
available and access to income during the search period will determine resource availability. 
While searching, individuals must weigh the risk of continuing to search for a job offering their 
“reservation wage” (the minimum wage that must be received to accept an offer of employment) 
against the risk of exhausting their resources and being forced to accept the next offer of 
employment. Other factors also affect the matching process, such as employer perceptions of the 
qualifications of potential employees, potential employee perceptions of job characteristics, and 


1 The ability of employers to do this for disability-related accommodations is limited by the Americans with 
Disabilities Act, which prohibits job-related discrimination against people with disabilities and requires that 
reasonable accommodation be provided for people with disabilities unless the difficulty or expense of 
accommodation would result in undue hardship for the firm. 
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the amount and nature of other information available to both parties in making their respective 
employment decisions. 


Income During the Job Search Process. Access to other sources of income during the job 
search process can affect the likelihood that an individual will accept a given job offer. Having 
other resources available allows the individual to be more selective, and can prolong a job search 
relative to if the individual had few or no other sources of income during the job search process. 
Because SSI, DI, and workers’ compensation benefits are not time limited, these programs might 
be expected to increase the time spent searching for employment among those seeking to return 
to work. When a person begins to receive DI, SSI or workers’ compensation, all else being equal, 
such an individual can afford to be more selective in searching for employment than a 
comparable person receiving temporary benefits, for example, under unemployment insurance, 
or one who is living on limited savings. 

Information. Information about the other party plays a key role on both sides of the 
matching process. Individuals need information about where job openings exist, the needed skills 
and other requirements of the job, job benefits, and the characteristics of the employer in order to 
find job opportunities that match their interests, capabilities and needs. Some employers will 
have an infrastructure and culture that are better able to meet the needs of people with specific 
types of disabilities relative to others. This information, however, can be difficult to discern 
during the interview process, and some job candidates with disabilities may be reluctant to 
question employers about accommodations or other disability-related needs for fear of 
jeopardizing their chances of receiving a job offer. In addition, employers might misrepresent the 
characteristics of the job or the firm. 

From the employer perspective, information is important for distinguishing who, among the 
pool of job candidates, is best suited for the position. Employers judge the potential productivity 
of job candidates based on both objective and subjective factors. Their assessments can be 
inaccurate if job candidates over-represent their qualifications, and/or if an employer’s 
assumptions about the impact of a disability on productivity are incorrect. As with any job 
candidate, employers have good reason to be skeptical of the candidate’s ability claims. The 
presence of a disability adds another dimension of uncertainty to the hiring decision, which 
might discourage some employers from hiring people with disabilities. 


d. Summary 

The above discussion suggests that a multitude of factors come into play in determining the 
employment outcomes of people with disabilities. It also suggests that many potential barriers or 
disincentives to work can arise for people with disabilities: 


• Poor health or functioning limiting the ability to engage in work or reducing the level 
of productivity 

• Inadequate education, skills, training, or job-related experience 

• Lack of reliable transportation to and from work 
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• Lack of specific supports needed while at work, or at home to prepare for/enable 
work 

• Loss of public or private cash and in-kind benefits as earnings and assets increase 

• Inaccessible workplaces and inflexible employment situations 

• Costly accommodations 

• Fear of discrimination and employer misconceptions of disability 

• Discrimination and employer misconceptions of disability 

• Lack of information about individual abilities and productivity 

• Insufficient wages or benefits offered with employment 

• Lack of information about employment-related supports and resources available 

• Lack of information about the impact of work on cash and in-kind benefits 

• Inadequate job search and interview skills or information 


These barriers are, presumably, among the factors intended to be addressed by the wide 
variety of federally sponsored initiatives designed to improve the employment of people with 
disabilities described in the remainder of this report. 


12 



II. FINDINGS 


A. NATURE OF THE INITIATIVES 

The 27 initiatives (shown in Exhibit 1) use a wide variety of strategies to address many of 
the barriers to employment faced by people with disabilities. Here, we describe some general 
observations regarding the nature of these initiatives. 

Most of the Interventions Focus on the Individual, Rather Than on Employers. The 

majority of the initiatives shown in Exhibit 1 are directed primarily at individuals, rather than 
employers. They are designed to increase the net value of work to the individual, increase human 
capital, make employment and employment-related resources more accessible, and/or assist the 
individual in the job search process. The disability-related tax provisions, Americans with 
Disabilities Act (ADA) provisions, Projects with Industry, and selected activities conducted by 
states under their Medicaid Infrastructure and Disability Employment Grants represent the few 
directed at employers and intended to increase the likelihood that employers will hire people 
with disabilities. 

Many Efforts Are Designed to Address System Complexity. Several of the initiatives are 
specifically designed to make information about disability and employment resources more 
readily available to people with disabilities and reduce the complexity of the service system from 
the perspective of individuals with disabilities. SSA’s Area Work Incentive Coordinators and 
Work Incentive Planning and Assistance initiatives are intended to provide better information to 
SSI and DI beneficiaries about the impact of work on disability benefits. The DOL Disability 
Navigator program is designed to provide information to people with disabilities using One-Stop 
Career Centers about the employment-related and other assistance programs and resources 
available in the community. The RSA/SSA Systems Change/State Partnership Initiative and 
CMS Medicaid Infrastructure Grant efforts provide support for states to look at their systems 
more broadly, and develop the means to better coordinate services and reduce system 
complexity. None, however, directly addresses the inherent complexity of the SSI and DI work 
incentive provisions. 

The Onus for Change Appears to Be on the State and Local Systems. Although all of the 

initiatives are in some way sponsored by federal regulations, agencies, and dollars, most are 
being designed and implemented by state and local entities. Even some of those directed 
specifically at beneficiaries of the federal DI program (for example, the Mental Health Treatment 
Study) rely on state and local providers to implement the intervention. This reflects that fact that 
most of the initiatives involve some form of service provision, and services are typically 
provided at the state and local level. 

There Are New Efforts to Combine/Coordinate Health and Vocational Supports. The 

importance of access to health insurance and maintaining health insurance coverage as 
employment status changes is evident in many of the initiatives (for example, Medicaid Buy-In 
programs, the Accelerated Benefits demonstration, Medicare provisions of the Ticket Act). In 
addition, a few of the initiatives go beyond simply providing access to health insurance because 
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they are designed specifically to combine and coordinate health and vocational supports in ways 
that promote employment (one of the treatment groups in the Accelerated Benefits 
demonstration, the Employment Intervention Demonstration Program, and the Mental Health 
Treatment Study). 

There Is Some Emphasis on Asset Development. Though somewhat meager, a few of the 
initiatives have an explicit focus on asset development and the impact of assets on continued 
program eligibility. Most of the existing Medicaid Buy-In programs permit higher levels of 
assets than would be permitted under other Medicaid categories. The Florida Freedom Initiative 
includes the use of individual development accounts, which permit savings in these accounts to 
be excluded from SSI eligibility determinations. 

Few Initiatives Focus on Early Intervention or Preventing Labor Force Withdrawal at 
Disability Onset. Most of the reviewed interventions focus on individuals who have already left 
the labor market and entered public programs. This might reflect, in part, the fact that many of 
the recent initiatives were developed in response to mandates in the Ticket Act, the provisions of 
which focus primarily on Social Security disability beneficiaries. It also reflects the fact that no 
federal agency has “jurisdiction” over people with disabilities until they become program 
participants. With the exception of the CMS demonstrations to maintain independence and 
employment, programs serving people with disabilities have not generally developed ways to 
provide supports that are intended to deter working people with disabilities from leaving 
employment and applying for public assistance benefits. The ADA, requiring employers to make 
accommodations when employees become disabled, and the laws permitting states to have 
Medicaid Buy-In programs have the potential to affect labor force withdrawal, but only to the 
extent that lack of access to accommodations and health insurance represent the primary barriers 
to ongoing employment. 

Most Initiatives Represent Small Changes to the Status Quo. Nearly all of the initiatives 
represent relatively small-scale or minor tweaks to the existing system of supports. Only the 
ADA represents a transformative change, in its broader purpose to guarantee equal opportunity 
and inclusion of people with disabilities in society. The Ticket Act and the Workforce 
Investment Act have elements that might have been used to bring about transformative changes 
in the way employment-related services and supports are provided to people with disabilities, but 
they have been implemented in ways that effect only relatively small changes to the status quo. It 
is possible, however, that the experiences gained from the many small-scale efforts currently 
under way, and the increasing attention being paid to employment issues for people with 
disabilities will provide a foundation for more fundamental changes in the future. 


B. ADEQUACY OF THE EVALUATION EFFORTS 

Although much useful information has been generated by the evaluation of the initiatives, 
many of these efforts have significant shortcomings or face specific challenges that limit the 
ability to draw firm conclusions about their impacts on employment. Below, we discuss the most 
important of these shortcomings and challenges. 

Data Is Often Unavailable or Inadequate. The lack of adequate data to evaluate the 
impact of the intervention on employment was an issue in reviewing many of the initiatives. In 
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some cases, the data needed to assess even basic outcomes were very poor or simply did not exist 
(for example, the three employer tax provisions). In other instances, administrative data from 
partnering agencies were needed, but they were unwilling or unable to cooperate due to resource 
or data confidentiality issues (for example, Systems Change/State Partnership Initiatives, 
Medicaid Buy-In). The general lack of data, in combination with the lack of an adequate 
comparison or control group (described below) are probably the two primary reasons why there 
is very little solid evidence on the impacts of the interventions on employment of the 
interventions. Substantial information exists regarding the implementation process and issues 
encountered, however, very little can be said with certainty about the impacts of the intervention 
on the employment of people with disabilities. 

Many of the Interventions Lack the Comparison or Control Groups Needed to 
Produce Rigorous Estimates of Their Impacts. The goal of an impact evaluation is to measure 
the causal effect of a program or intervention. That is, outcomes under the intervention must be 
compared to estimates of what they would have been under the counterfactual-usually meaning 
the status quo program or policy. Thus, it is important not only to observe the outcomes (or 
changes in outcomes) after the intervention, but to compare their levels (or changes) relative to 
those for a comparable group not affected by the intervention. Evaluations use two types of study 
designs to accomplish this — experimental (or randomized control group) designs, and quasi- 
experimental (or comparison group) designs. 

Experimental designs are considered the more rigorous of the two approaches. Random 
assignment of study participants (usually recruited volunteers) to treatment and control groups 
creates two groups that are statistically equivalent to one another. Barring significant violations 
of the experimental design (for example, some control group members receive the intervention) 
differences in outcomes between the two groups can be assumed to be entirely due to the impact 
of the intervention. 

Although some of the initiatives incorporate experimental designs (Benefit Offset, 
Accelerated Benefits, Employment Intervention Demonstration Program, Mental Health 
Treatment Study, four of the State Partnership Initiatives, and the four Demonstrations to 
Maintain Independence and Employment) a large majority of the interventions reviewed did not. 
This could be for at least two reasons. First, in the case of many of the initiatives we reviewed 
(for example, most provisions of the four federal Acts reviewed and the employer tax provisions) 
the initiatives were implemented in a manner such that random assignment was simply not 
possible. When Congress passes a law or creates a new program, it is no longer an experiment. It 
becomes a resource available to all in the target population, and, understandably, cannot easily 
be withheld from some individuals for purposes of evaluating its effect. However, in some cases, 
a quasi-experimental evaluation may be possible. We return to these types of evaluations below. 

The second reason for not implementing an experimental evaluation design is that the 
sponsors of the intervention chose not to do so. This can be for many reasons: the infrastructure 
and resources available are insufficient; the initiative is viewed as a “proof of concept” test , 2 


2 Proof of concept generally refers to a small, short, and/or incomplete test of a method or idea to demonstrate 
its feasibility and to provide some evidence that it might be capable of achieving the outcomes desired. 
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rather than a formal test of the impact of the intervention; the sponsors have ethical objections to 
withholding the intervention from control group members; the technical aspects of providing 
treatment to one group and not the other are problematic (for example, because members of the 
two groups might receive services from common providers or advisors); and/or the findings of a 
high quality evaluation of the intervention are not viewed as critical or worth the cost. From the 
information available to us for most of the initiatives, we cannot determine the relative 
importance of the preceding reasons in the cases where an experimental design was not pursued, 
but was technically possible. 

In several of the initiatives we reviewed (for example, State Partnership Initiative, Ticket to 
Work, Disability Employment Grants), a quasi-experimental study design was attempted to 
evaluate the impacts of the program or initiative. In these examples, the evaluators identified a 
comparison group of individuals who did not receive the intervention, but who, in many 
important ways, are very similar to the treatment group who received the intervention. Using a 
comparison group (rather than a randomized control group) approach can be problematic, 
however. The evaluators of several projects found that they could not identify comparison groups 
that did not differ systematically from the intervention groups. For example, in the State 
Partnership Initiative evaluation, Peikes and Sarin (2005) compared the impact estimates 
generated from the use of a sophisticated statistical method for constructing a comparison group 
with the estimates generated using an experimental control group. Using numerous variables, a 
propensity score matching technique was used to identify an appropriate comparison group. 
When this approach was applied to one of the state projects that had an experimental design, the 
evaluators found that the non-experimental method produced estimates of impacts that were 
different than those produced by the experimental design, and concluded that the non- 
experimental estimates were misleading. The apparent problem with the non-experimental 
design is that it was not possible to match comparison group members to treatment group 
members with respect to factors that motivated the participation of treatment group members in 
the demonstration. 

Another example is the Ticket to Work evaluation (Thornton et al. 2007), which attempted 
to exploit the phased rollout of the program and use a contemporaneous comparison group made 
up of beneficiaries in states where Ticket to Work had not yet been implemented. The evaluators 
found that the treatment and control states were not well matched. Differences across states in 
beneficiary earnings and benefit trends that existed before the Ticket to Work program rollout 
were so large that they obscured any effects that program might have had on these important 
outcomes. The problem, from an evaluation design perspective, appears to be that SSA selected 
states for the first rollout phase on the apparent readiness of state agencies and organizations for 
the rollout. 

Low Participation Rates in Some Studies Limited the Ability to Detect Impacts. In a 

few instances where formal experimental or quasi-experimental evaluation studies were planned, 
the studies encountered lower than expected participation in the intervention. When that happens, 
effects for those who participate become harder to distinguish from random noise. Low levels of 
participation can happen for a variety of reasons: the target population for the intervention is too 
small or defined too narrowly; the methods used to identify and recruit participants are 
inadequate; the study enrollment process is overly complex or burdensome; participants distrust 
the intervention or study; or the intervention is ill-conceived or unattractive to the target 
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population. The Florida Freedom Initiative had so few enrollees (just 35) that plans to evaluate 
the initiative were terminated. Although the Ticket to Work program has had over 200,000 
participants, that number represents only a little over one percent of the target population, and 
the vast majority of them would have received services under the counterfactual-the pre-Ticket 
to Work reimbursement program for state vocational rehabilitation (VR) agencies. The low 
participation rate has contributed to the inability of the evaluation to detect employment impacts, 
if indeed the program has generated any. 

Some Studies Pooled Data Across Interventions That Were Dissimilar, Making It 
Difficult to Draw Conclusions. Many grant programs offer the grantees wide latitude in the 
design of their interventions. Moreover, each grantee tends to serve a limited number of 
consumers (for example, Projects with Industry, Disability Employment Grants, Work Incentive 
Grants, State Partnership Initiative projects) making the sample sizes for each grantee too small 
to observe differences in outcomes. To offset this limitation, the national evaluations attempted 
to pool data across grantees, but because of the wide variation in populations served and types of 
interventions across grantees, pooling the data does not result in a useful assessment of the 
impacts of a particular intervention. 

Study Timeframes Are Short, but Target Outcomes and Behavioral Change Might Be 
Longer Term. Programs are often designed to produce long-term impacts, for example 
improving someone’s wage trajectory over the course of their working life or keeping them in 
the workforce. The temporary, and often short, timeframe of project interventions and 
evaluations often necessitate that the measurement of changes in wages or employment be made 
over periods as short as six months or one year (Disability Employment Grants, Projects with 
Industry, VR services). Programs that might demonstrate impacts in the long run may be found 
to have no significant benefits in short run evaluations. Conversely, short-term impacts identified 
in an evaluation might not persist beyond the evaluation period. 

Many Studies Assess the Impact of the Program on Service Users but Do Not Consider 
the Impact on the Larger Population of People with Disabilities. Many of the studies 
reviewed focused on measuring participant experiences, the impact of the program on participant 
outcomes, and in some cases, impacts on program expenditures. Broader analyses of the reasons 
for participation/nonparticipation and of costs and benefits external to the program were often 
not undertaken. This is likely for two primary reasons: the data needed to conduct broader 
participation analyses can be expensive to collect (the study must be able to identify relevant 
nonparticipants and collect information about their characteristics and reasons for 
nonparticipation). And if a rigorous impact evaluation for participants is not being conducted (as 
was the case in most of the studies reviewed) then there is no point in conducting a broader 
analysis of the costs and benefits associated with the program impacts. Many of the SSA 
initiatives represent at least partial exceptions to this limitation (for example, Ticket to Work, 
Mental Health Treatment Study, Accelerated Benefits, Benefit Offset). The target populations for 
these initiatives are well-defined (so non-participants can be easily identified) and formal impact 
evaluations, including some manner of cost/benefit component, are included in the evaluations. 

Measuring the Impact of Systems Change Activities Is Difficult. Fragmentation of 
services and funding sources is often cited as a barrier to employment supports. Many of the 
initiatives reviewed were intended to address this barrier by integrating funding, coordinating 
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services, formalizing referral services, and various other approaches (for example, Medicaid 
Infrastructure Grant, Workforce Investment Act, Disability Employment Grants, Work Incentive 
Grants, Disability Program Navigators, and the Systems Change/State Partnership Initiatives). 
However, evaluators have found it difficult to measure the impact of systems change activities 
on employment outcomes for several reasons. First, system quality indicators are generally not 
available. Concepts like attitudes, programmatic cohesion, and client understanding are hard to 
quantify, making the construction of systems change indicators difficult. Second, systems change 
occurs in many small increments that often take place over an extended period of time. 
Moreover, one change may have no effect until it is coupled with others. Isolating the effects of 
individual changes over the period of an evaluation is difficult, and made even more difficult by 
the fact that a variety of other factors affecting employment are occurring simultaneously (for 
example, changes in economic conditions, technology, the nature of work, and societal attitudes). 
For these and other reasons, evaluations of initiatives designed to change the overall support 
system have not provided convincing evidence of impacts on employment. These studies more 
often consist of process analyses that report on challenges, experiences, and “best practices” that 
have arisen from their efforts. Although, many of these best practices are idiosyncratic to the 
state system or specific target population, several grant programs (for example, Medicaid 
Infrastructure Grants, the Systems Change/State Partnership Initiatives, and several of the DOF 
grants) have developed successful strategies to disseminate each state’s lessons learned to other 
states. In several cases, these practices have been replicated in other states and grant programs. 
For example, the SSA/DOF Disability Program Navigator initiative was based on a best practice 
developed by one of the Work Incentive Grantees. 


C. CONCLUSIONS 

Considerable effort has been undertaken to implement and evaluate programs and special 
initiatives intended to improve the employment of people with disabilities. But are we making 
any progress? On the one hand, the employment rate of people with disabilities remains low, few 
of the initiatives reviewed were able to rigorously demonstrate positive impacts on employment, 
and many of the initiatives were not sustained after the special funding for them ceased. On the 
other hand, it does appear that system change is occurring. Greater attention is being paid to the 
incentives and issues surrounding employment for people with disabilities, including how the 
currently fragmented system of programs and supports can both hinder and help individuals, and 
how services from multiple sources might be coordinated to achieve better outcomes. SSA in 
particular has demonstrated a much stronger focus on employment in its disability programs in 
recent years than at any time in the past. CMS has actively promoted employment through 
support for Medicaid Buy-In programs and Medicaid Infrastructure Grant activities, many state 
Medicaid programs have taken the opportunity to expand coverage to workers with disabilities, 
many one-stops have substantially increased access to services for those with disabilities, and 
there are signs of increased cooperation between these programs and state VR agencies. Though 
there is evidence of progress, transformation is occurring slowly, and as GAO notes, the many 
federal programs serving people with disabilities continue to operate without a coordinated 
strategy or centralized mechanism for ensuring that programs and policies are aligned (GAO 
2008 a). 

How might we make greater progress toward improving the employment outcomes of 
people with disabilities? Two complementary strategies are suggested based on the findings of 
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this review: undertake bolder initiatives and generate better evidence on the impacts of the 
initiatives undertaken. Significant progress is not likely to occur unless we can move beyond 
incremental changes to a system that, according to many, is in dire need of modernization and 
transformation/ 1 Bolder initiatives would include ones that intervene early, before workers with 
disabilities have left the labor force and entered DI, and before youth with disabilities have 
entered a lifetime of dependency on SSI. Bolder initiatives would also provide incentives and 
mechanisms for multiple federal agencies to coordinate their efforts, pool resources, and strive to 
achieve common objectives. 

Better evidence of the impact of the initiatives undertaken is also needed. Experimental 
evaluation designs and strong non-experimental designs can generate compelling information to 
policymakers. Strong evidence of impacts is particularly important for justifying more 
transformative changes to existing programs and policies. 

The limited evidence of success makes efforts to improve the employment outcomes and 
self-sufficiency of people with disabilities seem daunting. But the experience of our nation’s 
welfare system gives cause for optimism. As noted by Butler (2007), there are many parallels 
between the experiences and factors that drove the 30- year transformation of the welfare system 
and what has been occurring more recently with our nation’s key disability programs. Changes in 
societal norms and expectations regarding the ability of single mothers to become employed and 
contribute to their families’ self-sufficiency, along with strong evidence of policy impacts 
derived from innovative policy experiments led to the significant changes in the welfare system 
embodied in the current Temporary Assistance for Needy Families (TANF) program and Earned 
Income Tax Credit for low-income parents. The focus of TANF is on providing supports and 
incentives that will help participants achieve greater self-sufficiency and reduce long-term 
dependency on public income supports. Societal norms and expectations surrounding the 
employment of people with disabilities have already changed, but we have yet to launch rigorous 
tests of highly innovative, transformative programs that improve the employment and self- 
sufficiency of people with disabilities. 


3 Shortcomings of the current system and the need for transformation have been noted by GAO 2003, NCD 
2005, Social Security Advisory Board 2006, Stapleton et al. 2006, and GAO 2008a. 
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III. DESCRIPTIONS OF INITIATIVES AND EVALUATION EFFORTS 


A. LEGISLATION 

1. American with Disabilities Act of 1990 
a. Intervention and Target Population 4 

The Americans with Disabilities Act (ADA) was signed on July 26, 1990. The ADA made 
discrimination based on disability illegal in all employment activities. The goal was to reduce 
some of the structural and institutional barriers people with disabilities faced in workplaces. By 
July 26, 1994, the ADA applied to all employers with 15 or more employees. 

The ADA considers a disability to be “a physical or mental impairment that substantially 
limits a major life activity,” or “if you have a history of such a disability, or if an employer 
believes that you have such a disability, even if you don’t”. The ADA is designed to prevent 
discrimination against people with disabilities, but the law also applies to people who are 
discriminated against because of their association with a person with a disability (for example, 
family members). The ADA applies to all aspects of employment, such as hiring, firing, benefits, 
pay, promotion, and such. 

The ADA also requires employers to make reasonable accommodations for employees with 
disabilities. These accommodations could be, for example, allowing flexible work schedules, 
making the work area physically accessible, or providing an interpreter. However, the ADA does 
allow for exceptions. If the modification would be too expensive or difficult then the employer is 
not required to make the change. Additionally, if any employee’s disability renders him or her 
unable to perform the essential functions of the job, the employer can refuse to hire the person. 
Inability to perform functions that are not essential to the job cannot be used as a basis for a 
decision to not hire an applicant with a disability. 


b. Evaluation Activities and Findings 

Numerous qualitative and quantitative assessments of the impact of the ADA on the 
employment of people with disabilities have been conducted since the implementation of the Act 
in 1992. Here, we provide a brief description of the general findings from a few of the more 
recent efforts. 

The empirical evidence regarding the effects of the ADA on the employment of people with 
disabilities is mixed. Acemoglu and Angrist (2001) used Current Population Survey (CPS) data 
to analyze the effect of the ADA on the employment of people with disabilities. They found that 
in the 21-39 age category, employment rates decreased for men with disabilities in 1993, and 


4 This section is based on information in U.S. Equal Employment Opportunity Commission (2005). 
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women with disabilities in 1992; both years correlate closely with the beginning of ADA 
enforcement. In their analysis, they controlled for the increase in the number receiving SSI. 
Interestingly, the largest decline in employment for people with disabilities was in medium-sized 
companies. The authors suggested that this is because small employers were exempt from parts 
of the ADA, while the larger firms could more easily absorb the costs associated with its 
enforcement. They also found that rates of employment dropped relatively more in states where 
the most ADA discrimination charges were filed — further suggesting that the ADA contributed 
to the lower employment rates. The employment rate effects were less for the 40-58 age group. 
The authors speculate that this is because people over age 40 were already protected by the Age 
Discrimination in Employment Act, and women were protected by Title VII. 

DeLeire (2000a, 2000b) came to similar conclusions as Acemoglu and Angrist (2001), 
through analysis of data from the Survey of Income and Program Participation (SIPP). After 
controlling for demographic characteristics, industry, and occupation, DeLeire found that 
employment rates for people with disabilities were steady until 1989, then began to fall in 1990 
and onward. DeLeire suggested that employers were not hiring people with disabilities in order 
to avoid the costs of accommodations and potential litigation based on the ADA protections. He 
noted that, while most accommodations are generally inexpensive, as others have shown, a 
substantial share are not — 12 percent cost more than $2,000 and 2 percent cost more than 
$20,000. He also noted that researchers underestimate the impact of accommodations that do not 
have a direct cost — such as flexible scheduling. 

The findings of these early studies have been challenged by many. Like most impact 
evaluations of new nationwide policies, they have to rely on non-experimental methods that are 
limited by the availability of comparison groups and data. In this case, the researchers needed to 
use the pre-ADA period as the comparison period, and data from surveys for that were conducted 
in both the pre-ADA and post- ADA period. The ADA’s implementation period occurred during 
a period when changes in other programs were potentially having an impact on employment of 
people with disabilities, as well as during a significant recession. Hence, these researchers faced 
the challenge of separating any effects of the ADA from the effects of other changes. They also 
had to use data that do not cleanly identify the population that the ADA’s employment 
provisions are applicable to and employment measures that were not arbitrary in important 
respects. 

Houtenville and Burkhauser (2004) challenged the conclusions of DeLeire and Acemoglu 
and Angrist that the ADA was responsible for the observed decline in the employment rate of 
people with disabilities. They used the model developed by Acemoglu and Angrist (2001), but 
additionally analyzed data for people with a two-period work limitation (those who reported 
work limitations each March, two years in a row), who are presumed to have a more long-term 
disability and are more likely to meet the definition of disability intended by the legislation. They 
find that the decline in employment for the two-period work limitation group begins well before 
the ADA, and the start of the decline appears more closely linked to the expansion of eligibility 
criteria for SSI and DI benefits that began in 1984. Goodman and Waidmann (2003) also argue 
this point, and suggest that the recession in the 1990s factored into the decrease in employment 
rates. Houtenville and Burkhauser also showed that the results were very sensitive to changes in 
the definition of employment. Kruse and Schur (2003) analyzed SIPP data and found that 
alternative definitions of disability produced different conclusions regarding the effect of the 
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ADA on employment, and suggested that any findings should be taken with a certain degree of 
caution. Hotchkiss (2004) analyzed SIPP and CPS data and concluded that the apparent decline 
in employment is partly due to nondisabled individuals who are not in the labor market being 
reclassified as disabled, rather than due to an increase in the labor force withdrawal of people 
with disabilities. She postulated that the increased reporting of disability may have occurred as a 
result of more stringent welfare reform requirements and the availability of more generous 
federal disability benefits. Jolls and Prescott (2004) exploited state variation in anti- 
discrimination laws prior to the ADA to estimate the impact of the ADA on employment in states 
with no pre-ADA law and states with laws against discrimination that, unlike the ADA, did not 
require employers to provide reasonable accommodations. They found immediate negative 
effects of the ADA on employment only in the states that had no pre-ADA law or had a law that 
did not require reasonable accommodations. They also found that the negative effects 
disappeared after a few years. 

Qualitative assessments of the impact of the ADA also find mixed evidence regarding its 
impact on employment. Most recently, the National Council on Disability (NCD) released a 
large report assessing the impact of the ADA (NCD 2007a). The study based its findings on a 
review of publicly available documents and data, stakeholder interviews and focus groups, and 
analyses of data from several of the NOD/Harris surveys of people with disabilities to assess 
changes in perceptions and attitudes over time. The report concludes there is evidence that 
people with disabilities are experiencing less discrimination and greater accommodation on the 
job, but they do not appear to be experiencing increases in hiring. The report also notes that 
many people with disabilities, employers, and businesses still do not understand major provisions 
of the ADA, particularly the employment provisions, and that this lack of understanding is 
reducing the effectiveness of the legislation. 


2. Balanced Budget Act of 1997 
a. Intervention and Target Population 

Under Section 4733 of the Balanced Budget Act of 1997 (BBA), states became able to 
expand Medicaid coverage of working people with disabilities by creating a new optional 
eligibility group whereby working individuals with disabilities meeting certain medical, income, 
and asset criteria may be permitted to “buy in” to Medicaid by paying a premium. Medicaid 
Buy-In programs are intended to help people with disabilities by providing access to health 
insurance to individuals who have relatively high earnings but do not qualify for Medicaid under 
one of the other eligibility provisions. Individuals are not required to be on SSI or DI to be 
eligible for the Medicaid Buy-In. However, the state must determine if the individual would be 
eligible for these programs if they were not working. The legislation allows states to provide the 
benefit to individuals with incomes up to 250 percent of the federal poverty standard. States are 
permitted to deduct certain expenses from income before applying the income test. SSI asset 
limits are also applied ($2,000 for an individual and $3,000 for a couple). 

The Medicaid Buy-In was designed to allow low-income workers with disabilities to 
purchase affordable health insurance, reducing a major incentive to withdraw from the workforce 
and seek SSI. Because Buy-In eligibility does not, in theory, depend on current or former SSI/DI 
participation, it weakens the incentive for a person with a disability to withdraw from the 
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workforce or to restrict earnings to receive health insurance benefits based on SSI or DI 
eligibility. There is still, however, an incentive to restrict earnings to remain within the Buy-In 
income limits, and the attractiveness of the program to potential participants will depend on its 
premium levels. 

The 1999 Ticket to Work and Work Incentives Improvement Act (Ticket Act) expanded 
state authority originally granted under the BBA to offer Medicaid Buy-In programs to working 
people with disabilities. As described further below, the Ticket Act eliminated eligibility 
restrictions on income and assets, and expanded eligibility to a “medical improvement'’ group — 
individuals whose medical conditions have improved and are determined to be no longer eligible 
for SSI or DI, but who still have a severe impairment. 


b. Evaluation Activities and Findings 

We describe the evaluation activities and findings to date associated with Medicaid Buy-In 
programs established under both the BBA and the Ticket Act in Section III.B.2 below. 


3. Ticket to Work and Work Incentives Improvement Act of 1999 
a. Intervention and Target Population 

Signed by President Clinton on December 17, 1999, the Ticket to Work and Work 
Incentives Improvement Act (Ticket Act) is intended to address a number of work disincentives 
inherent in the DI and SSI programs. We describe the nature of these provisions below. 


Ticket to Work and Self-Sufficiency Program 

Section 101 of the Ticket Act amended the Social Security Act to establish an SSA Ticket to 
Work and Self-Sufficiency Program (TTW). TTW provides disability beneficiaries who are 
appropriate candidates with a voucher, or ticket, to be used to obtain vocational rehabilitation or 
employment services. The Ticket can be assigned to any pre-qualified provider of these services. 
The provider is paid based on the earnings and benefit outcomes of the beneficiary. While a 
beneficiary is using a Ticket, SSA is prohibited from initiating a continuing disability review 
(CDR), the periodic review that determines whether a beneficiary is still medically eligible to 
receive disability benefits. 

The motivation behind TTW is to expand the employment and training opportunities of 
beneficiaries by expanding their access to a wide variety of vocational services. SSA had 
previously paid for services provided to DI and SSI beneficiaries by state VR agencies if the 
beneficiaries met a specific employment goal. TTW expanded the providers that beneficiaries 
could use to obtain SSA-funded services, but under more stringent outcome criteria. The goal 
was to allow market forces to reward providers who successfully move people with disabilities 
into work, through the use of consumer choice and performance-based reimbursement. The 
provision that suspends continuing disability reviews (CDRs) for individuals participating in the 
program aims to reduce the risk of benefit loss that a person faces when he or she enters a VR 
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program. The provision ensures that people will not avoid rehabilitation services for fear that 
their participation will make them ineligible for disability benefits. 


Expansion of Health Insurance Eligibility 

The fear of losing eligibility for public health insurance is frequently cited as a major 
disincentive to work for people with disabilities. Because health care costs are often extremely 
high for people with disabilities, the cost of health care can often outweigh the financial benefits 
of a job that does not provide comprehensive health insurance coverage. For many individuals 
with disabilities, particularly those who can work but are not capable of working a full-time job 
that would provide health benefits, employment is simply not a financially viable option. The 
health insurance provisions of the Ticket Act are intended to address this situation by improving 
access to health insurance coverage in the following ways: 


• Medicaid Buy-In Programs. Section 201 of the Ticket Act loosens restrictions on 
states regarding who is eligible to buy into the Medicaid program. Under this 
legislation, states have the option to eliminate all income, asset, and resource 
limitations for workers with disabilities who buy into Medicaid. States can also 
continue to offer the Medicaid Buy-In to workers with disabilities, even if they are no 
longer eligible for DI or SSI because of medical improvement. States are authorized 
to require individuals to pay premiums, or other cost-sharing charges, on a sliding 
scale. Section 203 of the Act also established a grant program to help states build 
infrastructures to support working individuals with disabilities. These grants became 
known as the Medicaid Infrastructure Grant (MIG) and the Comprehensive 
Employment Opportunity grant. Although intended to encompass a broader range of 
supports than just health care, the grants were initially tied to the development of state 
Medicaid Buy-In programs and the provision of expanded personal assistance 
services under Medicaid, and are administered by the Centers for Medicare and 
Medicaid Services. 

• Extended Medicare Coverage. Section 202 of the Ticket Act provides for the 
continuation of Medicare coverage for individuals formerly receiving DI benefits who 
leave the rolls due to work. Under previous law, the extended period of eligibility 
allowed DI beneficiaries to continue receiving premium-free Medicare Part A 
coverage for a total of four years after they returned to work. The Ticket Act extends 
this period for up to an additional four and one-half years. 

• Suspension of Medigap Policy. Section 205 of the Ticket Act allows workers with 
disabilities who have a Medigap policy — a commercial insurance policy that provides 
benefits supplemental to Medicare — to suspend the premiums and benefits of the 
Medigap policy if they have employer-sponsored coverage. This allows workers to 
take advantage of employer-sponsored benefits, an important incentive to work, while 
retaining eligibility for their Medigap policies in case their employment attempts fail. 
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Other Work-Related Provisions 


Several other provisions of the Ticket Act are design to promote employment among SSA 
disability beneficiaries: 


• Work Activity and CDRs. In addition to the limitations on conducting CDRs for 
individuals participating in the TTW program, Section 111 of the Ticket Act prohibits 
the use of work activity as a basis for review of an individual’s disability status. A 
beneficiary who is working is still subject to CDRs on a regularly scheduled basis, 
but the initiation of work cannot trigger such a review, and work activity may not be 
used as evidence that an individual is no longer disabled. This makes work activity 
less risky and presumably more attractive. 

• Expedited Reinstatement of Benefits. Section 112 of the Ticket Act allows for 
expedited eligibility determinations for applications of former long-term beneficiaries 
that have completed their extended period of eligibility. If an individual returns to 
work, earns enough to be ineligible for disability benefits for longer than the extended 
period of eligibility, 5 and is later unable to work, he or she will be able to return to the 
DI rolls without the lengthy eligibility process to which first-time applicants are 
subjected. Like the limitation of CDRs, this provision reduces the risk beneficiaries 
face when returning to work. 

• Work Incentives Outreach Program. Sections 121 and 122 of the Ticket Act directs 
SSA to establish a community-based work incentive planning and assistance program 
to provide accurate information related to work incentives to beneficiaries with 
disabilities. Under the program, SSA is to establish a competitive program of grants, 
cooperative agreements, or contracts to provide benefit planning and assistance to DI 
and SSI beneficiaries. Such grant programs are to include information on the 
availability of protection and advocacy services. The programs are also to conduct or 
fund ongoing outreach efforts and establish a corps of work incentive specialists 
within SSA who specialize in DI and SSI work incentives. In response, SSA 
established the Benefits Planning, Assistance and Outreach (BPAO) program, a 
cooperative agreement program with community-based providers, which later became 
known as the Work Incentive Planning and Assistance (WIPA) program. SSA also 
piloted the Employment Support Representative (ESR) position within SSA, which 
later became the Area Work Incentive Coordinator (AWIC) position. 


Research and Demonstration- Related Provisions 

In addition to the new programs and modifications to the DI and SSI programs described 
above, the Ticket Act included several research and demonstration-related provisions: 


5 The Extended Period of Eligibility (EPE) is a period of at least 36-months in which a DI beneficiary is 
eligible to receive DI cash benefits for any month in which countable earnings are below the SGA level. 
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• TTW Evaluation. The Ticket Act directs SSA to conduct an evaluation of the TTW 
program to assess the determinants of TTW participation, the characteristics of 
beneficiaries and providers who participate, the types of services provided and 
received, the employment outcomes of participants, beneficiary satisfaction with the 
program, and the costs and impacts of the program. 

• GAO Studies. Section 303 of the Ticket Act directed the Government Accountability 
Office (GAO) 6 7 8 to conduct several studies of various aspects of the DI and SSI 
programs: a study to assess existing tax credits and other disability-related 
employment incentives under Federal law; a study to evaluate the coordination of the 
DI and SSI programs as the programs relate to individuals entering or leaving 
concurrent entitlement, with a focus on the effectiveness of the work incentive 

o 

provisions of both programs; a study of the effect of the substantial gainful activity 
(SGA) level on employment; 9 and a study to assess the results of SSA’s efforts to 
conduct disability demonstrations authorized under prior law as well as under section 
234 of the Social Security Act. 10 

• Demonstrations to Maintain Independence and Employment. Section 204 of the 
Ticket Act appropriated funds to support states in conducting demonstrations 
designed to provide health insurance and supports to working individuals with 
physical or mental impairments that are reasonably expected to become blind or 
disabled without adequate medical treatment and supports. The expectation is that the 
early provision of adequate health insurance and medical supports to those who 
would otherwise lack such supports will allow individuals with potentially disabling 
conditions to improve their health and functioning and retain their attachment to the 
labor force. 

• DI $1 for $2 Benefit Offset Demonstration. Section 302 of the Ticket Act directs 
SSA to conduct a demonstration and to evaluate the effects of a $1 for $2 reduction in 
DI payments for earnings over a specified level. This “phase-out” of benefits will 
make the DI benefit and incentive structure more similar to that of SSI. It eliminates 
the earnings cliff currently faced by DI beneficiaries, where earnings beyond the SGA 
level eventually result in a total loss of benefits. The Ticket Act directs SSA to 
conduct an evaluation of the demonstration to determine the impact of the policy on 
induced entry into the DI program and the impacts on federal program expenditures. 


6 In 2004, the GAO’s legal name became the Government Accountability Office. Provisions that reference the 
GAO in the Ticket Act use the agency’s former name, the General Accounting Office. 

7 See GAO (2002c) for the completed study. 

8 See GAO (2002b) for the completed study. 

9 See GAO (2002a) for the completed study. 

10 See GAO (2005) for the completed study. 
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b. Evaluation Activities and Findings 


A number of evaluation activities have been initiated in response to the new programs and 
provisions implemented under the Ticket Act. In later sections of this report, we describe the 
evaluation activities and the findings to date associated with the following programs: 


• Demonstrations to Maintain Independence and Employment (Section III.B.l) 

• Medicaid Buy-In programs (Section III.B.2) 

• Medicaid Infrastructure Grants (Section III.B.3) 

• GAO study of the effectiveness of employer tax incentives to hire and accommodate 
people with disabilities (Section III.F.l) 

• Benefit Offset Demonstration (Section III.G.2) 

• BPAO/WIPA programs (Section III.G.3) 

• ESR/AWIC staff (Section III.G.4) 

• GAO study of the effect of SGA on employment (Section III.G.7) 

• Evaluation of the TTW program (Section III.G.8) 


4. Workforce Investment Act of 1998 
a. Intervention and Target Population 11 

In 1998, Congress passed the Workforce Investment Act (WIA) to unify a fragmented 
employment and training system, requiring states to provide most federally funded employment- 
related services through One-Stop Career Centers. The goals of WIA include improving both the 
quality of the workforce to sustain economic growth and productivity and competitiveness, and 
reducing dependency on welfare. 

Under WIA, states are required to create workforce development plans that describe how the 
state will meet the needs of major customer groups, including individuals with disabilities, and 
show how the plans will ensure nondiscrimination and equal opportunity. Each state’s governor 
is required to: 


• Establish a state workforce investment board to help design, implement, and provide 
oversight of the new workforce development system. The state board is responsible 
for organizing the service system to most effectively serve customers with multiple 
barriers to employment, including individuals with disabilities. 


11 The description of WIA is based on Silverstein (2000). 
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• Oversee the creation of local workforce investment boards to set policy for the local 
portions of the statewide workforce investment system. 

• Designate local workforce investment areas. The local workforce investment boards 
perform many of the same function as the state boards, but on the local level. 

• Develop one-stop delivery systems in all local areas: These delivery systems provide 
job search, job training, and occupational education programs. 


The one-stop centers provide core services to all individuals interested in learning more 
about the labor market or employment opportunities. Core sendees include determinations of 
eligibility for assistance; intake and initial assessment services; job search, placement, and career 
counseling; provision of program performance information and program cost information; 
provision of vocational rehabilitation services; assistance in establishing eligibility for Welfare- 
to-Work activities and education and training; and follow-up services, including counseling. 
These services are often self-directed, but individuals who experience difficulty or prefer to work 
with a staff member may receive “core assisted services.” 

Individuals who fail to find employment after the provision of core services can be eligible 
to receive intensive services. These services include comprehensive and specialized assessments 
of the skill levels and service needs; development of an individual employment plan; counseling 
and career planning; case management for those seeking training; short-term prevocational 
services; literacy activities related to basic workforce readiness; and out-of-area job search 
services. 

If the core and intensive services fail to lead to a job, the one-stop can offer the customer 
access to classroom or on-the-job training. Training is not an entitlement under WIA, and the 
statute states that if there are insufficient funds to provide training to all suitable customers, 
preference is to be given to low-income individuals. 

WIA specifically addresses the needs of people with disabilities by — 


• Mandating linkage of the State vocational rehabilitation systems to the State 
workforce investment systems 

• Streamlining the current State vocational rehabilitation systems 

• Improving delivery of services by providing more consumer choice, facilitating self- 
employment for disabled individuals, ensuring certain core services are available to 
all eligible individuals, and improving the clients' dispute resolution process 

• Mandating that one-stop centers and services be readily accessible to all Americans 


WIA requires that some agencies become partners in this system, including employment 
services, adult education, post-secondary vocational education, vocational rehabilitation (VR), 
Welfare-to-Work, and Community Services Block Grant recipients. In addition, state and local 
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boards can authorize or require other partners to participate in the one-stops. Other organizations 
serving people with disabilities that sometimes serve as partners at the one-stops include state or 
county mental health agencies, state mental retardation/developmental disabilities agencies, and 
community-based organizations. 

In 1999, DOL implemented nondiscrimination provisions of WIA that prohibited denying a 
qualified individual with a disability the opportunity to participate or benefit from WIA services 
or affording such individuals lesser, different, separate or segregated opportunities, or otherwise 
limiting a qualified individual with a disability in the enjoyment of a right, privilege, advantage, 
or opportunity afforded others. Under these provisions, states must also offer individuals with 
disabilities services in the most integrated environment possible. 

Hoff (2002) summarizes the nondiscrimination and comprehensive access requirements of 
the WIA regulations with respect to people with disabilities as follows: 


• People with disabilities have a right to use the services of the one-stop system. 

• One-stop career centers must be readily accessible to people with disabilities. 

• People with disabilities are entitled to reasonable accommodations and modifications 
when using one-stop services. 

• People with disabilities should not be automatically referred to the public Vocational 
Rehabilitation System. 


WIA was due to be reauthorized in 2003; however, the House and Senate could not reach an 
agreement on its terms (U.S. Department of Labor 2006). As of the drafting of this report, the 
attempts to reauthorize the Act continue. 


b. Evaluation Activities and Findings 

No formal evaluation of the impact of WIA on access to services or the employment 
outcomes of people with disabilities has been conducted. 12 Below, we briefly describe the 
general Government Performance and Results Act (GPRA) and Program Assessment Rating 
Tool (PART) indicators used to assess WIA performance, and also summarize the findings of 
two qualitative assessments of the extent to which one-stop services are accessible to and being 
used by people with disabilities. 


12 In June 2008, the DOL awarded MPR a contract to conduct an evaluation of the impact of WIA. The study is 
currently in the early design stages. 
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GPRA and PART Indicators 


WIA programs serving adults and dislocated workers utilize three employment-related 
outcome measures that states are required to report on a quarterly basis: (1) entered employment 
rate, (2) employment retention rate, and (3) average earnings. States are also required to provide 
a variety of other information about the characteristics of service users and the types of services 
they received, although this information is not used as part of the GPRA and PART performance 
assessment; only the aggregate outcomes are evaluated. Based on these outcome measures, the 
WIA met its 2005 targets (Exhibit 2). 


EXHIBIT 2 

WIA PART OUTCOME MEASURES 


Measure 

2005 Target 

2005 Actual 

Entered employment rate 

76% 

76% 

Quarterly earnings of those who are employed (average of first, 
second, and third quarters after the exit quarter) 

$3,400 

$4,044 

Employment retention rate 

81% 

82% 

Source: www.ExpectMore.gov 


The PART indicators are a subset of the 17 performance measures that state and local areas 
report to DOL. These measures include entry into unsubsidized employment, employment 
retention for six months, earnings six months after employment, attainment of appropriate 
credentials, and customer satisfaction. States and local areas that achieve or exceed their 
performance standards can receive additional funds. However, states and local areas can lose 
funding as well as the right to operate the WIA program if they fail to meet standards for two 
consecutive years (Holcomb and Bamow 2004). Two studies (described below) have suggested 
that the PART performance indicators act as disincentives for one-stops to serve customers with 
disabilities, because they are perceived to be less likely to find employment, and thereby 
negatively affect One-stop employment performance metrics (GAO 2004b, Holcomb and 
Bamow 2004). 


GAO (2004b) 

The GAO (2004b) conducted a study to address the following questions: 


• What have the DOL, states, and one-stop centers done to facilitate comprehensive 
access to the one-stop system? 

• What relationships have the one-stops established with disability-related agencies to 
provide services to persons with disabilities? 
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• What has DOL done to ensure that one-stops are meeting the comprehensive access 
requirements, and what factors have affected efforts to ensure compliance? 

• What is known about the employment outcomes of persons with disabilities who use 
the one-stop system? 


To address these questions, GAO conducted site visits to 18 local workforce investment 
areas in six states, and interviewed WIA officials, one-stop staff, and officials from organizations 
representing people with disabilities about issues and practices for providing programs, services, 
and activities to individuals with disabilities. In addition, they reviewed relevant documents from 
the DOL, as well as the WIA statute and regulations and other relevant statutory and regulatory 
provisions. 

To varying degrees, most of the sites visited by GAO had worked to implement architectural 
access requirements. One-stops, VR agencies, and other disability-related organizations in the 
community had established various relationships to provide services to persons with disabilities. 
GAO concluded that, although DOL has taken steps to ensure comprehensive access to one- stops 
by persons with disabilities, the efforts may not be sufficient because the DOL has not 
established a long-term plan for monitoring and enforcing the comprehensive access 
requirements. 

The report also notes that the ability to evaluate the employment outcomes of people with 
disabilities is limited by the extent to which disability data are collected by the one-stops and the 
overall data collection and reporting methods used under WIA. Although DOL has issued 
guidance that information about disability status must be collected from job seekers, disclosure 
by job seekers is voluntary. Finally, the report notes that because WIA performance levels and 
standards are tied to incentives and sanctions, there may be disincentives for one-stops to serve 
people with disabilities if their employment outcomes are likely to be poorer because there is no 
adjustment in the performance measurement system to account for differences in the populations 
served, including percentage of people with disabilities among job seekers utilizing one-stop 
services. 


Holcomb and Barnow (2004) 

In 2004, the Ticket to Work and Work Incentives Advisory Panel contracted with The Urban 
Institute to examine the extent to which people with disabilities are served through WIA’s one- 
stop system and to assess the system’s capacity to serve people with disabilities, both in terms of 
common barriers to access as well as promising strategies to improve service delivery. The 
study, conducted by Holcomb and Barnow (2004), based its findings on an analysis of selected 
WIA program data, a review of existing literature, and discussions with a wide range of 
individuals knowledgeable about different aspects of the one-stop system in connection with 
people with disabilities. 

Holcomb and Bamow (2004) note that the WIA program enrolls a number of people with 
disabilities; however, such individuals comprise a small proportion of the customers served. In 
program year 2002, only eight percent of individuals who exited the WIA program had identified 
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disabilities. Data limitations precluded tracking enrollment by beneficiary status, but the authors 
estimate that fewer than 2,400 exiters from the adult program in program year 2002 were SSI 
beneficiaries. The trend in services to people with disabilities is mixed. For the adult program, 
the total number of exiters increased each year, but the number of exiters with disabilities only 
increased between 2000 and 2001. Once enrolled in WIA, the services received for customers 
with disabilities are close to the figures for the overall population. Finally, exiters with 
disabilities have lower employment and earnings than other customers who exit the program, but 
they have greater earnings increases from the pre-program period. 

During the early years of WIA, anecdotal evidence suggested that the one- stops were not 
particularly accessible to customers with disabilities. According to Holcomb and Barnow (2004), 
these barriers included lack of physical access to the facilities, absence of appropriate hardware 
and software for customers with disabilities to access information on computers and in hard 
copies, and staff inexperience and lack of knowledge about how to identify and serve customers 
with disabilities. In addition, customers were generally expected to use the computers and 
printed material in a resource room on their own. This may have discouraged potential customers 
with disabilities from using the one-stop centers because, although special assistance was 
supposed to be available, it was not always offered, at least in part because staff found it difficult 
to identify people with disabilities with need for assistance. 

DOL and SSA established several grant programs designed to mediate these and other 
barriers for people with disabilities, including: 


• Customized Employment Grants funded by the DOL Office of Disability 
Employment Policy 

• Disability Program Navigators (DPN) funded jointly by the DOL Employment and 
Training Administration and the SSA Office of Program Development and Research 

• Work Incentive Grants (WIGs) funded by DOL Employment and Training 
Administration 


We discuss each of these programs separately in Section III.D below. 


B. CENTERS FOR MEDICARE AND MEDICAID SERVICES 
1. Demonstrations to Maintain Independence and Employment 
a. Intervention and Target Population 

Under the authority of the Ticket Act, CMS is conducting a set of demonstrations targeting 
workers who have physical or mental impairments that, without medical assistance, will likely 
result in withdrawal from the workforce. These demonstrations allow states to provide a health 
care package similar to Medicaid and a range of employment supports. 
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The Ticket Act identified the target recipients for the demonstrations as individuals with a 
specific physical or mental impairment who are reasonably expected, but for the receipt of 
medical care, to become blind or disabled as defined by SSA (Ticket Act, Section 204b). It was 
designed to offer medical care to those individuals who would otherwise need to leave the labor 
force and apply for disability benefits in order to gain access to health insurance and health 
supports. 

The first two rounds of solicitations for state proposals to design and implement 
Demonstrations to Maintain Independence and Employment (DMIE) occurred in 2000 and 2002. 
These solicitations focused on working individuals with chronic or progressive conditions whose 
conditions had not progressed to the point where they would meet SSA’s medical definition of 
disability. The four proposals submitted focused on progressive impairments such as HIV/AIDS, 
multiple sclerosis, and mental illness. Only the District of Columbia successfully established a 
viable program under these early solicitations. 

In response to the lack of interest in the DMIE on the part of states, during subsequent grant 
cycles, CMS clarified the definition of the target population to include any worker with a chronic 
or progressive condition who is at risk of withdrawing from the labor force and applying for 
Social Security cash benefits. In other words, it included individuals who might meet SSA’s 
medical definition of disability, but would not meet the work criteria for disability. CMS also 
provided states substantially more flexibility to develop a coordinated set of medical and 
vocational services to investigate the question of whether a program of medical assistance and 
other supports can forestall or prevent the loss of employment and independence due to a 
potentially disabling and medically determinable physical or mental impairment. Four additional 
states were awarded DMIE grants in the later rounds. The nature and status of the DMIE 
initiatives are shown in Exhibit 3. 


b. Evaluation Activities 

CMS contracted with the Research Triangle Institute (RTI) to evaluate the DMIE programs 
in the District of Columbia and Mississippi. However, because of low enrollment in the 
Mississippi program, it was discontinued. Thus, the RTI report evaluated only the program in the 
District of Columbia (Haber and Osber 2005). 

CMS contracted with MPR to conduct a national evaluation of the four states with viable 
programs established under the later solicitations (Kansas, Minnesota, Texas, and Hawaii). In 
contrast to the DMIE program in the District of Columbia, which did not plan for an 
experimental or quasi-experimental evaluation design, these states are operating programs that 
incorporate random assignment designs. Beginning in 2008, MPR will produce annual reports on 
the DMIEs, and in 2010, MPR will submit a final report to CMS on the national findings. The 
MPR evaluation will focus on estimating the impact of the initiatives on employment, access to 
and use of health care, and health status. The evaluation will be based on data collected by the 
states via survey, and maintained in state administrative systems. It will also use Medicaid claims 
data from CMS, earnings and disability program participation information from SSA 
administrative data, and qualitative information gained through site visits, interviews, and 
document reviews. 
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EXHIBIT 3 


SELECTED FEATURES OF THE DMIE INITIATIVES 


State * 

Enrollment Start Date 

Target Population 

Target and Actual Enrollment ** 

District of 
Columbia 

September 2002 

Employed individuals who are HIV- 
infected, with incomes at or below 
300% of poverty, assets below the 
Medicare Buy-In resource limit 
($4000 for individuals; $6,000 for 
couples), and are otherwise 
ineligible for Medicaid 

Target: Treatment: 420; Control: 
NA 

Actual: Treatment: 399 

Kansas 

April 2006 

Employed individuals enrolled in 
the Kansas Health Insurance 
Association high-risk pool 

Target: Treatment: 200; 

Control: 200 

Actual: Treatment: 189; Control: 
149 

Minnesota 

December 2006 

Employed individuals with serious 
mental illness from five counties 

Target: Treatment: 1,125; 
Control: 375 

Actual: Treatment: 330; Control: 
103 

Texas 

April 2007 

Employed adults enrolled in the 
Harris County Hospital District 
medical program for uninsured 
residents with either (1) severe 
mental illness or (2) behavioral 
health diagnosis co-occurring with a 
physical diagnosis 

Target: Treatment: 800 
Control: 625 

Actual: Treatment: 351; Control: 
280 

Hawaii 

June 2007 

Employed adults with diabetes who 
reside in the city and county of 
Honolulu 

Target: Treatment: 356 
Control: 178 

Actual: Treatment: 0; Control: 0 


Sources: Department of Health and Human Services, Centers for Medicare and Medicaid Services. Program 
Announcement: Demonstration to Maintain Independence and Employment. Issued March 17, 2006; Gimm and 
Weathers (2007); and Gilman et al. (2008). 

In April 2002, Mississippi implemented a DMIE program for people with HIV infection, but because of low 
enrollment, transitioned all enrollees out of the demonstration by September 30, 2007. Rhode Island and Louisiana 
also were approved under earlier solicitations, but neither state implemented a program. Iowa received a planning 
grant in May 2008 to assess outcomes of a program to help adults with mental illnesses who have been recently 
discharged from correctional facilities. 

Actual enrollment figures for the District of Columbia are as of December 2006. Enrollment figures for Kansas, 
Minnesota, Texas, and Hawaii are as of December 2007. 


The later-round grantees were also required to include independent evaluations of their 
programs as part of their grant activities. Each state has contracted with its own evaluator to 
conduct these activities. 
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c. Progress to Date and What Will Be Learned in the Future 


The DMIE program in the District of Columbia covers HIV-positive individuals with 
incomes up to 300 percent of the federal poverty level who work at least 40 hours per month or 
120 hours over a three-month period. It differs from the demonstrations implemented under the 
subsequent grant cycle because it does not provide enhanced case management services to DMIE 
enrollees and the services do not focus on employment support. In addition, most of the enrollees 
were already receiving adequate HIV medications and HIV-related care through the AIDS Drug 
Assistance Program and Ryan White clinics, so the demonstration primarily represented a 
change in payment source (Haber and Osber 2005). 

The RTI evaluation of the District’s program focused on program implementation and 
operations. It found that within two years of its inception, the program had reached its enrollment 
cap of 420 individuals and a waiting list for the program was instituted. Program participants and 
case managers noted that enrollment in the DMIE improved access to treatment for non-HIV- 
related conditions because participants received coverage for the full range of Medicaid services. 
The study did not assess the impact of the program on health or employment outcomes. 

Because of its design, the demonstration in the District of Columbia is unlikely to provide 
information on the impact of providing improved access to health care on the employment of 
individuals with HIV/AIDS. It is possible that the RTI evaluators could use individuals on the 
waiting list as a control group and collect information about employment outcomes from both 
groups; however, we have no information about RTFs plans for the District of Columbia 
evaluation, or the potential for producing impact estimates. The demonstration in the District of 
Columbia (and other states) is, however, likely to produce useful information about the demand 
for and feasibility of offering health coverage combined with intensive case management to 
workers at risk of leaving the labor market because of a disability. 

The MPR national evaluation is still in its early stages. As it is based on a random 
assignment design, it is expected to produce rigorous estimates of the impact of the DMIE 
interventions on health care access and utilization, health status, employment, and SSEDI 
program participation and benefits. A potential limitation is that the small sample sizes or low 
levels of enrollment at some sites could limit the ability of the evaluation to detect some impacts. 
The first interim evaluation findings are expected to become available in early 2009, and the 
final evaluation findings approximately one year later. 


2. Medicaid Buy-In Programs 
a. Intervention and Target Population 

The 1999 Ticket to Work and Work Incentives Improvement Act (Ticket Act) expanded 
state authority originally granted under the 1997 Balanced Budget Act (BBA) to provide 
Medicaid coverage to workers with disabilities who, because of income and assets, would not 
otherwise qualify for Medicaid coverage. Both the BBA and the Ticket Act authorized states to 
charge premiums for this coverage, and thus, allow working individuals with disabilities to “buy 
in” to Medicaid. 


36 



The program was developed in response to evidence that many individuals with disabilities 
face the choice of private employer-based insurance, which might not cover needed services, 
prohibitively expensive coverage in the private market, and the being uninsured. As a result, they 
often rely on public health insurance. However, because of the link between health insurance 
(Medicaid and Medicare) and eligibility for disability benefits (SSI and DI), which require that 
the individual not be working at substantial levels, this system creates a strong work disincentive. 
The Buy-In programs are designed to break the link between benefits and health care and thus 
remove the work disincentive inherent in the public programs. 

As of July 2008, 35 states were operating a Medicaid Buy-In program. Across all states, 
nearly 190,000 working-age individuals with disabilities have enrolled in a state Medicaid Buy- 
In program at some point between 1997 and 2006 (Gimm et al. 2008). 


b. Evaluation Activities 

CMS contracted with MPR to document the outcomes of the program. MPR has produced 
multiple reports and policy briefs that describe enrollment trends, participant characteristics, 
earnings, and Medicaid expenditures. 

The annual reports for 2002, 2003, and 2004 (Ireys et al. 2003, White et al. 2005, Black and 
Ireys 2006) were based on summary data submitted by states derived from their various 
databases. The state-submitted data allowed CMS to track participation in the Buy-In program, 
however, the states did not have access to SSA or Medicare program data, and therefore lacked 
information that could be important for program evaluation, such as the participant’s impairment, 
SSI and DI benefits, and Medicare expenditures. In addition, states could submit earnings data 
only from their Unemployment Insurance program systems, which are not as comprehensive or 
as accurate as federal earnings data available from IRS records. 

To address these shortcomings, CMS worked closely with SSA to modify existing 
interagency data-use agreements such that person-level data from Medicare, Medicaid, and SSA, 
including IRS records available to SSA, could be used to evaluate the program. 13 This approach 
provides more complete and consistent data among states and reduces their reporting burden (Liu 
and Ireys 2006). The states provided CMS with individual-level enrollment data through “finder 
files” that provided CMS with identifying information about each Buy-In participant (for 
example, social security number, date of birth). MPR then linked the finder files to Social 
Security, Medicare and Medicaid data sources. The combined data file has information on Buy- 
In participants who had been an SSI or DI beneficiary between 1996 and 2005, including data on 
their ethnic background, primary disabling condition, benefit receipt, earnings, and dates of SSI 
or DI program enrollment as well as Medicare and Medicaid expenditures. MPR used these data 
in subsequent reports to present a snapshot of the program participants in 2005 (Ireys et al. 
2007), report the characteristics of high earners (Gimm et al. 2007), compare annual earnings 
among first-time Buy-In participants before and after their initial enrollment in the program (Liu 


13 Only SSA staff are allowed access to the IRS data. Hence, analyses that required use of the IRS earnings 
data were completed by SSA staff. 
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and Weathers 2007), and assess the determinants of employment and earnings (Gimm et al. 
2008). 

In addition to the MPR studies, there have been several smaller scale research activities. The 
Office of the Assistant Secretary for Planning and Evaluation contracted with George 
Washington University to assist stakeholders in designing and implementing Medicaid Buy-In 
programs. They developed several reports based on the experiences of nine “early implementer 
states.” While not formally “evaluations” of the program, these reports describe the program 
design and how these designs affect enrollment, costs, and state fiscal exposure. The reports also 
describe the programmatic, fiscal, and political context in which these design decisions were 
made (Folkemer et al. 2002a and 2002b, Jensen et al. 2002). 

Many states have conducted evaluations of their Buy-In programs using funds from their 
CMS Medicaid Infrastructure Grants, including, for example, California (Jee and Menges 2003), 
Iowa (Iowa Department of Human Services 2005), Maine (Clark et al. 2003), , and Wisconsin 
(APS Healthcare 2003). On behalf of the Ticket to Work and Work Incentives Advisory Panel, 
Goodman and Livermore (2004) reviewed the information that existed at that time regarding 
evidence of the effectiveness of Medicaid Buy-In programs in promoting the employment of 
people with disabilities. 


c. Findings 

The major findings from the MPR studies noted above include the following: 


• Program design and enrollment 

> There is considerable variation across states in the eligibility criteria and cost 
sharing policies used by Buy-In programs (Folkemer et al. 2002a, Goodman and 
Livermore 2004, Ireys et al. 2007). These variations have consequences for the 
ability of particular Buy-In programs to promote employment, especially among 
DI beneficiaries (Goodman and Livermore 2004, Black and Ireys 2006). 

> States have adjusted the eligibility requirements and operational features of their 
Buy-In programs over time for various reasons, including to strategically expand 
or restrict the pool of potential applicants; to allow current participants to earn 
more money; to alter the structure of premium payments for purposes of 
increasing revenues or increasing participation; and/or to alter grace periods in 
order to make it easier or more difficult for participants to stay enrolled if they 
lose their jobs (Ireys et al. 2007). 

> There is large variation in enrollment among states, from a low of one participant 
in South Dakota’s newly implemented program, to more than 9,000 in 
Massachusetts in 2006 (Gimm et al. 2008). 

> Certain program features have been shown to be associated with participant 
employment outcomes. Shorter grace periods (work stoppage provisions), work 
verification requirements, and higher earned income limits were positively 
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associated with both the likelihood of employment and the level of earnings 
(Gimm et al. 2008). 


• Participant characteristics 

> Most (71 percent) Buy-In participants in 2006 were DI beneficiaries (Gimm et al. 
2008). 

> The most common primary disabling condition of Buy-In participants is severe 
mental illness and other mental disorders. About 30 percent of all Buy-In 
participants in 2006 had these conditions (Gimm et al. 2007). 

> Most 2006 participants were white (76 percent) and over age 40 (58 percent) 
(Gimm et al. 2008). 


• Health care costs 

> Average Medicaid expenditures per member per month (PMPM) were $1,076 
among Buy-In participants in the 22 states with a program as of 2002; average 
PMPM Medicare expenditures were $391. 

> Medical expenditures vary by type of impairment. For example, PMPM Medicaid 
and Medicare combined expenditures in 2002 were $1,042 among Buy-In 
participants with musculoskeletal disorders, and $1,695 among those with mental 
retardation. 

> Medical expenditures also vary by receipt of federal disability benefits. PMPM 
Medicaid and Medicare combined expenditures in 2002 ranged from $1,179 
among Buy-In participants receiving only SSI benefits to $1,491 among those 
receiving only DI benefits. 


• Employment and earnings 

> In 2006, 69 percent of Buy-In participants were employed and had earnings 
during that year. The likelihood of employment was significantly associated with 
age (negative), S S I/D I program status (negative), certain disabling conditions, and 
as noted above, certain features of the programs (Gimm et al. 2008). 

> Broadly speaking, the data indicate that Buy-In participants engage in levels of 
employment that allow them to maintain their eligibility for DI benefits. Of the 43 
percent of Buy-In participants nationally who had earnings reported in state 
unemployment insurance data systems, about 7 in 10 had earnings below the SGA 
level (Black and Ireys 2006). 

> Approximately 40 percent of new Buy-In participants increased their earnings, 
with adjustments for inflation, after they enrolled in the program. The median 
inflation-adjusted increase in their annual earnings was $2,582. This is substantial 
relative to the average annual pre-enrollment earnings of $4,844 (Liu and 
Weathers 2007). 
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> The rate of earnings growth differed based on participant characteristics and 
across states: 65 percent of participants under age 21 increased their earnings and 
the percentage steadily declines to 47 percent for those ages 21 to 44, 33 percent 
for those ages 45 to 64, and 30 percent for those ages 65 and older (Liu and 
Weathers 2007). 

> The share of people with increased earnings post-enrollment differs substantially 
across states, from 58 percent in Nebraska to 20 percent in New Mexico. These 
differences may be in part the result of factors such as state labor markets and 
policy environments. However, they may also be due to state- specific program 
features that affect the characteristics of participants (Liu and Weathers 2007). 


The goal of the Medicaid Buy-In program is to increase employment among people with 
disabilities by eliminating the risk that employment will cause an individual to lose access to 
public health insurance. Although studies indicate that Buy-In enrollees increase their earnings, 
on average, after enrollment in the Buy-In, no strong evidence exists that these programs caused 
the increase in employment. In rigorous experimental evaluations of employment programs for 
other populations, it is common to find that control group as well as treatment group earnings 
increase because those who volunteer to participate are seeking to increase their earnings, and 
many continue to do so, regardless of whether they are assigned to treatment or control. Hence, it 
might be that increases in earnings observed for Buy-In participants would have occurred, in 
whole or in part, in the absence of the Buy-In. 

Conducting a rigorous evaluation of the impact of Buy-In programs on employment is made 
difficult by the absence of data on suitable comparison groups (Goodman and Livermore 2004). 
As the programs were implemented statewide at a given point in time, within-state comparison 
groups are not available. It is possible that a study design incorporating pre-post comparisons 
across states with and without a Buy-In over time might detect impacts of the program on 
employment, but the large degree of variation across states, both in the nature of their Buy-In 
programs and in factors that likely affect labor market outcomes in general, would make the 
detection of relatively small impacts unlikely. 

An important outcome of the MPR studies is that they led to establishment of an interagency 
agreement between CMS and SSA for linking administrative data from the two agencies for 
purposes of program and policy evaluation. Although MPR has used the linked data mainly for 
descriptive analyses, linking data across these programs holds promise for conducting other 
future program evaluations. 


3. Medicaid Infrastructure Grants 
a. Intervention and Target Population 

The Medicaid Infrastructure Grant (MIG) program was established by the Ticket to Work 
and Work Incentives Improvement Act of 1999 and is administered at the federal level by CMS. 

Congress authorized the MIG program for 11 years beginning in fiscal year 2001. States 
apply for funding annually but there is no maximum to the number of years for which a state is 
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eligible. States are expected to use the funds first to change the Medicaid program by 
establishing Medicaid Buy-In programs and other Medicaid enhancements that would support 
employment (such as personal assistance services). As states achieve these goals, they can use 
the funds more broadly to “promote linkages between Medicaid and other employment-related 
service agencies and to develop a comprehensive system of employment supports for people 
with disabilities” (CMS 2007). 

States have used the grants primarily to provide education and outreach for Medicaid Buy-In 
programs and other work incentives; conduct needs assessments and cost estimates for 
expanding personal assistance services; conduct ongoing monitoring and evaluation of Buy-In 
programs; convene leadership councils; and build interagency relationships. As of July 2008, 40 
states had MIGs. 


b. Evaluation Activities 

The primary outcome of the MIGs is the establishment of the Medicaid Buy-In programs. 
The evaluations of the Buy-In programs are described in the previous section. There is no federal 
evaluation of other MIG activities. Grantees are, however, required to report grant-related 
activities to CMS on a quarterly basis, and innovative practices are shared among states via two 
technical assistance centers — the National Consortium for Health Systems Development and the 
Center for Workers with Disabilities. In addition, the Law, Health Policy and Disability Center at 
the University of Iowa’s College of Law, through a grant from the Rehabilitation Services 
Administration in the U.S. Department of Education, provides ongoing reports about MIG 
activities. 14 


c. Progress to Date and What Will Be Learned in the Future 

Although there have been no evaluations and no formal identification of “best practices,” 
CMS included in the request for MIG proposals several steps to facilitate information sharing 
among states, including the following: 


• MIG grantees must provide state-to-state technical assistance either by joining one of 
the two technical assistance networks or developing their own plan. 

• Grantees must attend the annual technical assistance conferences. 

• Grantees must report their activities to an on-line reporting system designed to 
“facilitate the sharing of technical assistance and contacts”. 

• States are required to post all technical assistance products purchased with grant 
funding and make them available for use by other states. 


14 See http://www.Medicaidbuyin.org. 
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CMS provided funds to establish a northeast regional network of MIG grantees, 
which meets regularly 




C. DEPARTMENT OF EDUCATION 
1. Federal/State Vocational Rehabilitation Services 
a. Intervention and Target Population 

Title I of the Rehabilitation Act of 1973 authorizes a federal/state Vocational Rehabilitation 
(VR) program to provide services to persons with disabilities so that they may prepare for and 
engage in gainful employment. It provides federal funds to help people with disabilities become 
employed, more independent, and integrated into the community. The federal funds are chiefly 
passed to state VR agencies that directly provide services such as guidance, counseling, and job 
placement, as well as purchase services such as therapy and training from other providers. In 
2007, the U.S. Department of Education provided about $2.9 billion in VR grants to the states 
and territories based on a formula that considers the state’s population and per capita income. 15 
The Act generally requires states to match federal funds at a ratio of 78.7 percent federal to 21.3 
percent state dollars. Each state and territory designates a single VR agency to administer the VR 
program, except where state law authorizes a separate agency to administer VR services for 
individuals who are blind (NCD 2007b). 

Section 101(a)(5) of the Act requires VR agencies to give priority to serving individuals 
with the most significant disabilities, many of whom benefit from the specialized services 
provided under VR (about 86 percent of the individuals served are those with significant 
disabilities). 


b. Evaluation Activities 

There have been several efforts to evaluate the VR program: 


• Annual GPRA and PART indicators 

• The Longitudinal Study of the Vocational Rehabilitation Services Program 
(LSVRSP) 

• Several GAO studies 


1 5 http ://www. ed . gov/ about/overview/budget/ statetables/index.html. 
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GPRA and PART Indicators 


The U.S. Department of Education (ED) collects information about all individuals who exit 
each state VR agency’s program during a particular fiscal year, as reported by each state VR 
agency. The record for individuals exiting the program includes information such as whether or 
not they became employed, the weekly earnings and hours worked for individuals if they exited 
the VR program with employment, the types and costs of services they received, and 
demographic factors, such as impairment type, gender, age, race and ethnicity, public benefit 
receipt, and income from work at the time of application. ED also collects summary information 
on agency expenditures in a number of categories from each state VR agency. ED tracks seven 
types of case closures, which can be collapsed into individuals who, 1) exited without 
employment during the application phase; 2) exited without employment with limited services; 
3) exited without employment after receiving services under an employment plan; and 4) exited 
with at least 90 days of employment after receiving services under an employment plan. 

With this information, ED reports the following measures on the PART evaluation: 


• Percent of state VR agencies (excluding VR agencies for the blind) that assist at least 
55.8 percent of individuals receiving services to achieve employment 

• Percent of state VR agencies (excluding VR agencies for the blind) that assist at least 
85 percent of individuals with employment outcomes to achieve competitive 
employment (employment in an integrated setting at/or above the minimum wage) 

• Percent of state VR agencies (excluding agencies for the blind) for which at least 80 
percent of the individuals achieving competitive employment are individuals with 
significant disabilities 

• Percent of general and combined state VR agencies that demonstrate average cost per 
employment outcome between $6,000 and $16,500 

• Percent of general and combined state VR agencies that demonstrate an average 
annual consumer expenditure rate of at least 83 percent 


LSVRSP 

In 1992, RSA initiated a study to assess the performance of the federal/state VR program on 
a broad spectrum of outcome measures. Between 1995 and the end of 1999, the Research 
Triangle Institute (RTI) collected data on a nationally representative sample of more than 8,500 
state VR consumers who were followed over a period of three years from application to closure 
to post-service outcomes. 16 The data collected include both survey data and data abstracted from 


16 The study implemented a multistage design that involved selection of a random sample (with probability 
proportional to size) of 40 local VR offices (located in 32 state VR agencies in a total of 30 states), and among those 
offices, a sample of 8,500 applicants and current and former consumers of VR services. The study implemented a 
cohort design that involved randomly selecting 25 percent of the sample from all persons at application to VR, 50 
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case files. The study’s database has much more detail on VR consumers than what is available 
through RSA administrative data regarding client characteristics, services received, and work 
outcomes. 

RTI collected data through a series of surveys with participants and a process of abstracting 
information from VR records. The baseline interview conducted at the time of entry into the 
study obtained information on work history, functioning, vocational interests and attitudes, 
independence and community integration, and consumer perspectives on their VR participation. 
Administration of a follow-up survey for three subsequent years varied according to the 
individual’s stage in the VR process at the time of interview. The records abstraction, which was 
conducted when the consumer entered the study, then continued quarterly until program exit, 
generated information on consumer characteristics and detailed information about services. 

In addition to collecting data about the consumers, RTI collected data about the agencies 
serving the study participants through mail surveys of office managers, counselors, and other 
office staff, using a form regarding state policies and procedures. These instruments were 
administered at initiation and termination of the study’s data collection activities, with annual 
updates from the local office manager surveys (Hayward and Schmidt-Davis 2003a). 

The study was designed to evaluate the VR program based on a conceptual framework 
where outcomes (earnings, employment, consumer satisfaction, community integration, and 
other economic and non-economic outcomes) are a function of the following: 


• The characteristics of applicants and consumers, including demographics, functional 
level, work history and earnings, interests and motivations, and type and severity of 
disability. 

• Services and service costs, including equipment, adaptive devices, supplies, 
professional services, and other RSA resources such as counselor and staff time. 

• Local economic and population characteristics, including the urban or rural nature of 
the area, the availability of jobs, and the prevalence of work disability in the state 
population. 

• The organizational culture and resources in the local agency office, such as internal 
organization and management philosophy. The organizational culture is viewed as an 
influencing factor on both quality of services and resulting service outcomes. 


The design did not include a method to determine what employment outcomes would have 
been in the absence of VR services, presumably because it would be problematic to identify a 
suitable comparison group. 


(continued) 

percent of the sample from all persons who were already accepted for and receiving services, and 25 percent of the 
sample from among all persons at or after they exited VR services. 
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GAO 2007 


In 2005, GAO reported that, based on the RSA 911 data, the employment rates for VR 
clients, particularly for SSA beneficiaries, varied substantially among VR agencies. As many as 
68 percent of SSA beneficiaries exited VR with employment in some states compared to as few 
as 9 percent in other states (GAO 2005). Based on these findings, Congress requested that GAO 
conduct a study to determine what might account for the wide variation across states in the 
employment outcomes of VR consumers who are DI and SSI beneficiaries. The study was not 
intended to be an impact analysis. It was designed to examine the extent to which (1) differences 
in VR agency outcomes for SSA beneficiaries continued over several years and across different 
outcome measures, (2) differences in VR agency outcomes were explained by state economies 
and demographic traits of the clientele served, (3) differences in VR agency outcomes were 
explained by specific policies and strategies of the VR agencies, and (4) ED’s data allowed for 
an analysis of factors that account for differences in individual-level (as opposed to agency- 
level) outcomes (GAO 2007). 


c. Findings 

GPRA and PART Indicators 

The PART assessment concluded that the VR program is performing adequately based on 
the annual performance goals. The performance measures focus on whether certain short-term 
objectives have been achieved, but because there is no counterfactual, they do not indicate the 
extent to which the program contributed to that achievement. They serve purely as descriptive 
performance benchmarks. 


LSVRSP 

RTI developed four primary evaluation reports (Hayward and Schmidt-Davis 2002, 2003a, 
2003b, and 2005) with descriptive statistics about the characteristics of VR clients (types of 
disabilities, functional status, receipt of financial assistance etc), the services they use and their 
outcomes (employment status, earnings, health insurance, community integration, independence, 
etc). 18 The evaluators used basic univariate and multivariate approaches to address five 
questions. We briefly summarize their findings regarding each question. 

1. What short- and long-term economic and non-economic (for example, independent 
living, community integration) outcomes do VR applicants and consumers achieve as a 
result of their participation in VR? 


17 http://www.ed. gov/rschstat/eval/rehab/standards.html?exp=l 

18 Additional interim and special topic reports were also developed for the study. See 
http://www.ed.gov/policy/speced/leg/rehab/eval-studies.html. 
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On all measures of economic outcomes, VR consumers who achieved a competitive 
employment outcome fared better than those who achieved a non-competitive employment 
outcome and those who exited without an employment outcome. For example, at the third annual 
follow-up, 78 percent of persons exiting into competitive employment were still working, 
compared with 70 percent of those exiting into non-competitive employment, 37 percent of those 
who received services but exited VR without an employment outcome, and 40 percent of those 
who were eligible but dropped out before receiving VR services. 

On measures of non-economic outcomes, consumers with competitive employment 
outcomes reported less frequent use of a variety of services and were less likely to report that 
their disability restricted their ability to participate fully in social and community activities 
compared to consumers with other outcomes. They were also more likely to believe that VR 
contributed to their success. 


2. What characteristics of individuals with disabilities affect their (1) access to and receipt 
of VR services, and (2) short- and long-term outcomes? 


Among all VR applicants, 84 percent were offered VR services. Of those who were offered 
services, 88 percent received services. The remaining 12 percent chose to exit the program prior 
to receiving services. Of those who received services, at the end of the data collection period, 17 
percent were continuing to receive VR services, 45 percent had achieved employment outcomes, 
and 21 percent had exited VR after services without an employment outcome. Among those with 
an employment outcome, 78 percent were working at jobs in the competitive labor market. The 
remaining 22 percent held non-competitive jobs. 

Using a multivariate approach, RTI found that the probability of each of these outcomes was 
positively or negatively correlated with a number of consumer characteristics, such as type of 
disability, psychosocial characteristics, reliance on other government programs for support, work 
history, career interests, motivation, and demographic characteristics. 


3. To what extent does receipt of specific VR services contribute to successful consumer 
outcomes? 


Overall, consumers who received VR services averaged 12 services during their 
participation in the program. The most frequent services delivered to consumers were 
medical/physical functional evaluation (62 percent), cognitive/psychological assessment and 
services (35 percent), employment development services (33 percent), postsecondary education 
(33 percent), and miscellaneous support services, such as transportation or maintenance (38 
percent). The type of services that a client received was a function of disability type and 
employment goal. 

Controlling for differences in consumer characteristics, a number of specific VR services 
contributed to the likelihood of an employment outcome and a competitive employment 
outcome. These included job placement, supported employment, on-the-job training, 
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independent living services, driver training/licensing, tools/uniforms/equipment and 
business/vocational training. Receipt of psychosocial counseling was associated with a decrease 
in the probability of an employment outcome. The consumer’s rating of the quality of the 
consumer-counselor relationship was also a significant determinant of employment outcomes. 


4. In what ways and to what extent do local environmental factors influence VR consumer 
services and outcomes? 


Each VR agency operates in a different environment in terms of general population 
characteristics, the employment environment, and community resources. Although some aspects 
of the local environment affected the likelihood that consumers would receive certain services 
and achieve employment outcomes, the identified features of the environment were much less 
important than consumer characteristics as determinants of these outcomes. 


5. In what ways and to what extent do the operations, resources, and organizational climate 
of VR agencies influence consumer services and outcomes? 


Local offices vary in terms of caseloads, financial resources, office policies and practices, 
and office culture. While a number of client characteristics were associated with the type or 
amount of services a consumer received, office factors were the primary determinant. 

The LSVRSP study has several limitations. Although the study controlled for differences in 
individual characteristics of VR clients and found some indication of the relationships among 
services and outcomes, the study design did not include a comparison or experimental control 
group. Thus, the evaluators could not conclude that specific services caused particular outcomes 
as there is no information about what would have occurred in the absence of VR services. 

Another limitation is that, because the study findings reflect the study’s data collection 
period, which occurred between 1995 and the end of 1999, they do not reflect recent changes that 
might have affected VR services and outcomes, such as WIA and the Ticket Act. 

One of the strengths of the LSVRSP is the rich data that was collected. Cornell University 
has made these data easily accessible and researchers have made use of the data for specific 
studies. Lor example, Capella-McDonnall (2005) identified predictors of competitive 
employment outcomes for blind or visually impaired VR consumers; Stapleton and Erikson 
(2005) analyzed the impact of SSI and DI recipiency on employment outcomes; and Homa 
(2005) analyzed the impact of vocational evaluation on VR outcomes. 


GAO 2007 

The GAO (2007) study found that agencies varied widely across different outcome measures 
including employment rates and annual earnings. Locusing on earnings outcomes in a 
multivariate model, the study found that much of the variation in state agency earnings outcomes 
could be explained by state economic conditions and the characteristics of the agencies’ clients. 
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Together, state unemployment rates and per capita income levels accounted for roughly one-third 
of the variation across state agencies in the share of SSI and DI beneficiaries with earnings 
during the year after exiting VR. The demographic profile of clients also accounted for some of 
the variation across states in terms of the likelihood that SSI and DI beneficiary VR clients 
would leave the disability rolls — women, older clients, and clients with mental illness or visual 
impairments were less likely to leave the disability rolls. 

A few agency practices appeared to be associated with positive earnings outcomes. For 
example, state agencies with a higher proportion of state-certified counselors had better 
employment outcomes. 

GAO was not able to determine which factors might account for differences in earnings 
outcomes at the individual level, in part because RSA’s administrative data lacked important 
information. Although the RSA administrative data has extensive client-level data, it does not 
systematically collect data that research has linked to work outcomes such as detailed 
information on the severity of the client’s disability — data that some state agencies 
independently collect for program purposes. Further, other key data are self-reported and might 
not be verified by state agencies. 


2. Projects with Industry 
a. Intervention and Target Population 

The Projects with Industry (PWI) program was designed to create and expand job and career 
opportunities for individuals with disabilities in the competitive labor market by engaging the 
participation of business and industry in the rehabilitation process. It is funded and administered 
by the Department of Education, Office of Special Education and Rehabilitative Services 
(OSERS). 

Since its inception in 1968, the program has provided grants to nonprofit organizations, 
educational institutions, private corporations, state VR agencies, Indian tribes, labor unions, trade 
associations, and other organizations to help individuals with disabilities obtain competitive 
employment. 19 All grantees are required to provide job development, job placement, and career 
advancement services as well as any required support services. Most PWIs also provide a range 
of optional services such as job readiness training, job skill training, and post-placement 
assistance. 

PWIs must also establish Business Advisory Councils (BACs) that include representatives 
of private industry, organized labor, individuals with disabilities and their representatives, state 
VR agency representatives, and others. BACs are intended to identify job and career availability 
within the community, identify the skills needed to perform those jobs, and prescribe for 
individual PWI participants an appropriate training or placement program. Some BACs also raise 


19 Rehabilitation Act of 1973, as amended. Title VI, Part A, Section 611. 
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the general awareness of the employment potential of persons with disabilities and of the PWI 
project (Tashjian 2003). 

In 2006, there were 79 PWI programs that served a total of 7,512 clients (U.S. Department 
of Education n.d.). PWIs vary considerably in terms of the types of individuals they serve. 
Participants are referred to individual projects from a wide variety of sources; traditionally a 
majority of participants are referred from the VR program. Eligibility criteria for PWI services 
are the same as for the federal/state VR program (Tashjian 2003). 


b. Evaluation Activities 

There have been several efforts to evaluate the PWI program: 


• Annual GPRA and PART indicators 

• An evaluation conducted by RTI (Tashjian 2003) 

• Studies conducted on specific programs, of which we describe the methods and 
findings of one (Fabian 2007) that assessed the employment outcomes of a very large 
sample of program participants 


GPRA and PART Indicators 

PWI grantees are required to submit primary and secondary performance indicators to 
OSERS annually. The primary indicators include placement rate into competitive and average 
increase in weekly earnings. Secondary indicators include cost per placement; placement rate 
among people with significant disabilities; and placement rate among those who were previously 
unemployed. These indicators are measured against a target established by OSERS. Targets are 
set based on expected project outcomes and take into consideration past program performance 
and factors such as grantee experience. For example, new grantees are expected to produce fewer 
outcomes in the first year of the grant. To receive continuation funding for the third or any 
subsequent year of a PWI grant, a PWI project must pass both primary indicators and at least two 
of the three secondary indicators. 


RTI Evaluation 

During 2000-2003, the U.S. Department of Education contracted with Research Triangle 
Institute (RTI) to “examine the role and performance of the PWI program as one component of 
the broader set of employment-related services available to individuals with disabilities, with a 
particular focus on the extent to which PWI projects fulfilled their intended goal to create and 

90 

expand job opportunities for individuals with disabilities at the project level” (Tashjian 2003).“ 
To address this question, the RTI report described the following: structure and operations of PWI 

20 Previous evaluations of the PWI program were conducted in 1985 and 1994. 
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projects; the relationship of the PWI program to the employment community; coordination 
between the PWI and federal/state VR programs; the validity and reliability of compliance 
indicators; the characteristics of individuals served by PWIs including their demographic and 
disability characteristics; types of PWI services received; and, employment outcomes obtained. 

RTI gathered data through two mechanisms: (1) a survey of all PWI projects funded as of 
October 2000, and (2) on site data collection at 30 randomly selected PWI projects. The on-site 
data collection included interviews and focus groups with BAC members, local VR staff, 
employers, and Workforce Investment Board chairs. The onsite data collection also included 
abstracting data from a randomly selected 20 percent of case files (584 individuals) whose 
participation in the PWI program ended in FY 2001. RTI weighted these data to represent the 
universe of PWI users and used the data to describe the characteristics, services, and outcomes 
for all former PWI participants. They compared these PWI measures to measures for VR 
consumers based on RSA-911 data (case service records) from those VR districts most 
proximate to the 30 selected PWI projects (Tashjian 2003). 


Fabian (2007) 

One recent study (Fabian 2007) assessed the employment outcomes of participants in a 
particular PWI program, the Marriott Foundation’s Bridges from School to Work Program 
(Bridges). The goal of the Bridges program is to provide competitive paid work experience for 
special education youth who are transitioning out of high school to adult life. Since the 
program’s inception in 1990, Bridges has served more than 10,000 youth in six major urban 
locations: San Francisco, Los Angeles, Chicago, Atlanta, Philadelphia, and Washington, DC. 
Bridges offers a standardized one-semester vocational intervention program consisting of three 
phases: (1) career counseling and job placement; (2) paid work experience with training and 
support provided by a Bridges employer representative; and (3) follow-along support and 
tracking of student participants. 

Fabian (2007) assessed the postsecondary employment outcomes of transition-age youth 
with disabilities who participated in Bridges from 2000 to 2005. The study explored factors that 
distinguished the students who obtained employment and the nature of the jobs that are secured 
using a database of 4,571 students enrolled in the Bridges program containing demographic 
information (race, age, gender, disability), background information (special education services 
received, whether the student received SSI), previous employment experience, and whether the 
student had identified a post-school career goal. 
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c. Findings 

GPRA and PART Indicators 


According to the annual GPRA indicators, on average, PWI projects were close to meeting 
the employment targets during 2004 and 2005, and exceeded the earnings increase targets in 
each year from 2003-2005 (Exhibit 4). The projects fell short of the employment targets during 
2003. 


EXHIBIT 4 


GPRA PERFORMANCE MEASURES FOR THE PROJECTS WITH INDUSTRY PROGRAM 



2003 


2004 


2005 


GPRA Indicator 

Actual 

Target 

Actual 

Target 

Actual 

Target 

Percentage of individuals served who 
were placed into competitive 
employment 

54.2% 

62.4% 

61.5% 

62.7% 

51.9% 

63.0% 

Percentage of previously unemployed 
individuals served who were placed 
into competitive employment 

54.0% 

63.0% 

65.5% 

64.0% 

62.4% 

65.0% 

Average increase in weekly earnings 

$242 

$231 

$247 

$233 

$253 

$238 

Cost per placement (federal grant 
funds only)* 

NA 

NA 

$3,139 

NA 

$3,014 

NA 


Source: U.S. Department of Education (2006). 

* The PWI data collection instrument was revised for FY 2005 reporting which resulted in a significantly lower 
reported placement rate compared to previous years. 


RTI Evaluation (Tashjian 2003) 

We briefly summarize the key findings of the RTI evaluation below. 

Structure and Operations. Individual PWI projects vary considerably with respect to 
scope, type of consumers served, priority areas addressed, types of services provided, and other 
factors. These variations tend to reflect the availability of other resources in the community and 
other programs available at the grantee organizations. 

Relationship of the PWI Program to the Employment Community. PWI projects also 
varied in the extent to which the BACs fulfilled their statutory requirement. The vast majority of 
BACs, according to survey data, pursued their legislated functions of identification of job 
openings and career availability (99 percent), and identification of necessary skills for those jobs 
(92 percent); somewhat fewer BACs actually prescribed participant training or placement 
programs (77 percent). Many PWIs had engaged businesses through other mechanisms such as 
formal agreements or long-standing relationships with one or more area employers or staffing 
agencies. The evaluators questioned whether the statutory role of the BAC was reasonable given 
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the voluntary nature of BAC services, turnover in BAC membership, and the infrequency with 
which most BACs convene. 

Coordination Between the PWI and Federal/State VR Programs. The relationship 
between VR and the PWI also varies considerably. While some projects obtained 100 percent of 
their participants through VR agency referrals, others served few or no VR consumers, and still 
others (school-to-work projects) served individuals with disabilities prior to their receipt of VR 
services. Individuals served by both programs tend to receive placement assistance only through 
the PWI program, with any training usually provided or at least funded by VR. PWI programs 
tend to emphasize job placement and follow up with participants and employers. Employers 
identify this follow-up as the reason they believe the PWI program is a better source of qualified 
employees compared to the VR services program or private employment agencies. 

Validity and Reliability of Compliance Indicators. About one-quarter of the projects did a 
poor job in collecting and maintaining the data required to report values for the performance 
indicators. The most difficult aspect was tracking former participants’ employment status over 
time. 

Demographic and Disability Characteristics of Individuals Served by PWIs: RTI found 
that individuals served by the PWI program, averaged over all grantee sites, do not differ much 
from those served by VR. Although each PWI project typically serves a specific subset of the 
disability population, when aggregated across all sites the population looks similar to that of VR. 

Types of PWI Services Received. PWI services most frequently included job placement 
(62 percent), job readiness training (59 percent), and job development (42 percent). Less 
frequently received services included job skills training (25 percent), supportive services (17 
percent), worksite modifications (2 percent), and career advancement services (2 percent). On 
average, participants received PWI assistance for 8.7 months; persons who obtained employment 
following PWI participation averaged of 9.6 months; and persons who exited without 
employment averaged 6.5 months. 

Employment Outcomes Obtained: RTI found that the data collection practices of the PWI 
projects limited their ability to formally evaluate the program’s impact on employment. However 
RTI found the following: 


• The outcomes of the PWI program, with respect to the percentage of persons served 
who exited into employment and the average hourly earnings of those individuals, are 
comparable to those of the VR services program. 

• Various combinations of program scope, type of consumers served, priority areas 
addressed, and types of services provided appear to be equally effective. 
The percentage of persons who obtained employment (as calculated from survey 
data) varied little when examined by these variables. 

• Although difficult to quantify, RTI noted that PWI effectiveness is likely in part a 
function of how well the project complements other grantee programs and the extent 
to which projects coordinate their services with those of other local programs, 
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including especially VR. RTI found that PWI participants who had received VR 
services more often obtained employment than PWI participants who had not 
received VR services. 


The RTI evaluation was not able to meet the goal of measuring whether the PWI program 
created and expand job opportunities for individuals with disabilities or whether involvement by 
the business community leads to better employment outcomes because of several limitations: 


• The programs varied so significantly, even in the core goal of engaging businesses 
and industry in the rehabilitation process, that very little can be learned about 
effectiveness of the approach by aggregating data across programs. 

• The data collection practices of the PWI grantees undermine the ability to accurately 
measure their achievements. 

• The programs were implemented with no control or comparison groups so the 
counterfactual is not known. 

• There is a great deal of overlap between the PWI programs and the VR program so it 
is difficult to separate the effect of PWI from that of VR. 

• Several functions of the BAC (for example, identifying careers and the corresponding 
needed skills) are now functions of the local workforce investment board under the 
Workforce Investment Act of 1998 (WIA). 


In interviews with employers, the RTI study identified potential strengths of the PWI 
program compared to VR including better candidate screening to ensure a good job match and 
better follow-up with employers and former participants. It is not clear whether these practices 
are replicable or have been replicated. 


Fabian (2007) 

Of the 4,571 youth participating in the Bridges program from 2000 to 2005, 68 percent 
secured jobs after completing the career counseling and career assessment components of the 
Bridges intervention. The average rate of competitive job placement was 68 percent across all 
years, while the average wage for all employed youth was $6.91 per hour. A number of 
individual characteristics and experiences were associated with the likelihood that a youth with 
disabilities obtained a job: girls were significantly less likely to secure jobs than boys; those with 
prior paid vocational experience were more likely to secure jobs; and those receiving SSI were 
less likely to secure jobs than those not receiving SSI. 

The study findings suggest that rehabilitation counselors should begin working with special 
education personnel as early as possible to formulate career-related activities and interventions, 
even prior to student entry into secondary schools. This directive corroborates other studies, 
which demonstrate that longer exposure to career and related interventions increases the 
likelihood of positive outcomes. The study was limited by the lack of a comparison or control 
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group to assess what the outcomes would have been in the absence of the program. The findings 
also are not necessarily representative of youth with disabilities in general, as selection bias is 
likely. Students who enrolled in Bridges might represent a group of students with disabilities 
who expressed a desire to participate in a vocational program prior to exiting school, and 
represent a subset of youth with disabilities who exhibit characteristics associated with academic 
persistence and secondary labor market employment aspirations. 


3. RSA/SSA Systems Change/State Partnership Initiative 
a. Intervention and Target Population 

Under the State Partnership Systems Change Initiative (SPI), SSA and RSA funded a 
combined total of 18 demonstrations in 17 states over the 1998-2004 period to identify, 
implement, and evaluate new strategies to promote employment opportunities for SS 1/D I 
beneficiaries, as well as recipients of other types of public supports. Twelve of the projects were 
funded primarily by SSA and the other six projects were funded primarily by the RSA. DOL and 
the Substance Abuse and Mental Health Services Administration provided supplementary 
funding and support. 

The amount and type of activities conducted by the SPI projects varied considerably 
between the SSA-funded demonstrations and the RSA projects. The SSA-funded demonstrations 
developed intensive, specialized interventions targeted toward a small group of beneficiaries, 
implemented under the limitations of a formal evaluation design. These projects served SSI and 
DI beneficiaries exclusively. Although the SSA projects were permitted to pursue systems- 
change activities, the programs used the most of their resources on research interventions and 
allocated relatively few resources to systems change efforts. In contrast, the six RSA-funded 
projects focused more on systems change efforts, both at the local and state level. They were not 
restricted to serving only SSA beneficiaries and as a result, the states targeted their programs to 
people with disabilities who receive public assistance benefits such as TANF, General 
Assistance, and/or Social Security disability benefits (Kregel 2006). 

Five of the 18 projects targeted people with severe mental illness exclusively, and an 
additional 3 projects targeted people with mental illness along with people with other disabilities. 
The remaining 10 projects targeted people with any type of disability. A number of the projects 
required participants to be enrolled for services at their state’s VR agency. 

The interventions implemented by the projects addressed the following service and policy 
gaps: 

Benefit Policies. Most of the projects offered benefits planning and assistance programs to 
help beneficiaries understand and take advantage of SSA work incentives. Four states tested 
waivers to SSI regulations designed to make employment more attractive to beneficiaries. These 
SSI waivers included provisions that permitted working beneficiaries to keep more of their 
benefits (benefits were reduced by $1 for every $4 above the applicable earnings disregard, 
instead of $1 for $2), to accumulate more savings and assets, and to be exempt from being 
continuing disability review triggered solely because of their participation in SPI (Peikes and 
Paxton 2003). 


54 



Access to Health Insurance. Many of the SPI projects were instrumental in the 
development and implementation of Medicaid Buy-In programs in their states, which were 
designed to allow beneficiaries to work without fear of losing medical benefits, and to provide 
coverage for some medical care, such as prescription drugs 21 and personal services, not covered 
by Medicare. 

Service System Barriers. Most projects tried to improve coordination among state agencies 
and the various organizations that provide employment supports to people with disabilities. A 
common approach was to include disability-related services in DOL’s one-stop centers. Some 
programs developed initiatives to foster interaction among state agencies that share responsibility 
for encouraging work among people with disabilities (Peikes and Paxton 2003). 

Human Capital and Personal Barriers. The SSA projects developed programs to help 
beneficiaries obtain skills that would help them compete in the labor market. These included 
case management services to help participants obtain needed services and supports, and 
placement assistance programs that provided participants with job placement and support 
services to help them find and maintain employment. Services were also provided through j ob 
sendee vouchers that enabled beneficiaries to obtain vocational services from a vendor of their 
choice. Two projects also tested ways to use peer support to help beneficiaries deal with the 
world of work. The type and intensity of services varied significantly across projects (Peikes and 
Paxton 2003). As the RSA projects were more generally focused on systems change, they did not 
provide these types of direct services to individuals. 

Employment Market Barriers. A few of the projects provided education, outreach, and 
direct incentives to encourage employers to hire more beneficiaries with disabilities. Some 
projects also used non-SPI funding for initiatives to promote employer awareness of the abilities 
and employment potential of people with disabilities (Peikes and Paxton 2003). 


b. Evaluation Activities 

SSA and RSA funded a SPI project office at Virginia Commonwealth University (VCU). 
VCU and its subcontractor, MPR, conducted the national evaluation. The evaluation was 
designed to identify the mix of services that were most effective in promoting employment, and 
identify services that were especially valuable for a particular participant group (Agodini et al. 
2002). Because the RSA-funded projects focused on systems change rather than direct services, 
the core and supplemental analyses described below are for the SSA-funded projects only. 

In addition to the national evaluation, each SPI project was required to develop an internal 
evaluation plan. Of the 12 SSA-funded states, four randomly assigned participants to an 
intervention or control group. The project office synthesized these evaluations in two reports 
(Peikes and Sarin 2005; Kregel et al. 2005). 

The state evaluations examined whether the projects accomplished the following: 


21 The SPI projects were implemented prior to implementation of the Medicare Part D drug benefit. 
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• An increase in employment rates and earnings 

• A decrease in the proportion receiving Social Security benefits and the amount of 
Social Security benefits received 

• An increase in participants' income (defined as earnings plus Social Security benefits) 


The national evaluation implemented a four-part study that combined multiple data sets and 
methods to produce several estimates of the effects of the state project interventions. It then used 
qualitative information about project implementation to synthesize those estimates and to 
develop an understanding of the relative performance of the projects. The four components of the 
national SPI evaluation include: 

Core Evaluation. The core evaluation component compared key outcomes of the 
beneficiaries who participated in the SSA-funded projects to outcomes of a comparison group 
selected to match the participants in terms of the characteristics of the areas in which they live, as 
well as their demographic characteristics, prior labor market experiences, and prior benefit 
receipt. The core evaluation used only SSA administrative data and income tax data so that the 
data and approach could be applied consistently across projects. 

Supplemental Evaluation. The projects collected data on beneficiaries at study enrollment 
and on a quarterly basis. These included detailed data about participants’ characteristics, their 
receipt of project services, and their employment and benefit receipt outcomes. As part of the 
supplemental evaluation, MPR produced two reports: 


• Characteristics of Participants Enrolled Through March 2003. Based on an analysis 
of data that state projects collected from participants at intake, the evaluators 
characterized the participants by demographics, type of disabling condition, 
education, employment experience, and use of public assistance programs (Peikes and 
Paxton 2003) 

• Services Delivered Through March 2002. Using information collected from 
participants on a quarterly basis about the types of services they used, the number of 
hours associated with each service, the use of work incentives, and other programs, 
the evaluators analyzed the type and intensity of services used and use of work 
incentives and Medicaid Buy-In programs (Deke and Peikes, 2003) 


State Projects’ Own Evaluations. SSA required each project to use its own evaluation 
design and data sources to describe the project’s implementation and to assess the project’s 
impacts. All of the projects completed final reports. Ten of the 12 projects provided final 
estimates of program impacts. MPR researchers synthesized the findings from SSA-funded 
projects in Peikes and Sarin (2005). 

Implementation and Synthesis Analysis. The SPI Project Office at VCU documented and 
analyzed the interventions fielded by the projects. This information provided context for the 
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evaluators to understand the ways in which the projects changed the services available to 
participants, as well as the context in which each project operated. 


c. Findings 

Generally, the SPI projects providing direct service interventions were able to meet their 
enrollment targets. From January 1999 through September 2003, they had roughly 10,300 
participants. Individual project enrollment ranged from 181 to over 2,000 participants, with an 
average enrollment of 582 (Peikes and Paxton 2003). 

The process evaluations indicated that the SPI projects had an effect on system change 
within states and provided replicable strategies that could be implemented in other states (Kregel 
et al. 2005). Some examples include: 


• The SPI projects led the way in the establishment of a nationwide system of Benefits 
Planning Assistance and Outreach (BPAO) Projects, 22 with many staff involved in the 
on-going training provided to these projects. 

• Several SPI projects were instrumental in facilitating the development and/or 
implementation of Medicaid Buy-In programs, at first through the Balanced Budget 
Act and later through the Ticket Act. 

• The model for the Disability Program Navigators initiative within the one-stop career 
center system that is currently being administered by the Employment and Training 
Administration of DOL was initially developed through the Colorado RSA-funded 
SPI project. 

• In a number of SPI projects, the use of benefits planning and assistance services by 
the state VR agency became a “routine” component of service delivery for SSA 
beneficiaries. 

• Multiple SPI projects demonstrated effective strategies for coordinating the efforts of 
employment service projects with local SSA field office staff. 


Findings regarding the outcomes and impacts of the SPI projects are mixed. The projects 
seemed to increase the proportion of participants who attempted to work. However, there is 
mixed evidence of the effects of SPI on average earnings and little effect on benefits (Peikes and 
Sarin 2005). 

For several reasons, the evaluators expected the impact on employment and earnings to be 
small to modest in the short-term. First, while participants faced many barriers to employment, 
most projects focused on one or two barriers, and so the projects were expected to be most 
successful for people who faced only those barriers. Second, two earlier SSA demonstrations 

22 Now called Work Incentives Planning and Assistance (WIPA) projects. 
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with more intensive services than provided under SPI (the Transitional Employment Training 
Demonstration and Project Network) generated only modest increases in participants’ 
employment and earnings. Third, the SPI program was fielded in an environment where many 
other services were available from state VR agencies, community organizations, and the BPAO 
projects. BPAO was implemented during SPI in some states because of the efforts of SPI project 
staff. As a result, some of the comparison group members received benefits counseling and 
employment services similar to those being offered by SPI. 

Testing the Validity of Comparison Groups 23 

In the core evaluation, MPR recognized that a beneficiary’s decision to participate in the SPI 
project might be related to the participant’s observable and unobservable demographic or human 
capital characteristics. Thus, they expected that the beneficiaries who participated in SPI would 
differ from the average SSI/DI beneficiary in terms of their motivation to work, work histories, 
and other characteristics that might be related to employment and earnings outcomes. To account 
for these differences, MPR used propensity score matching (a statistical matching technique) to 
develop a comparison group comprised of a subset of SSI/DI beneficiaries who were similar to 
SPI participants along 250 characteristics. Multiple statistical tests confirmed that the propensity 
score match yielded comparison groups that were well matched to the SPI participants on the 
observed characteristics. 

Using three state projects that had implemented random assignment study designs (New 
Hampshire, New York, and Oklahoma), MPR analyzed the validity of the propensity score 
comparison groups by comparing the impact estimates based on random assignment with 
estimates based on the propensity score comparison groups. They found that although the 
propensity score comparison groups were similar to SPI participants along the entire spectrum of 
available characteristics, the estimates based on the propensity-score comparison group were 
quite different than those based on the randomized control. The former suggested that the SPI 
interventions increased earnings by between $970 and $5,600 a year, whereas the latter showed 
that the interventions had no impacts, or even negative impacts on earnings. MPR ultimately 
concluded that non-experimental propensity score matching method could not provide reliable 
estimates of the SPI program effects, most likely because the comparison groups did not match 
the participant group on unobserved characteristics such as motivation to work and informal 
support from family, friends and others (Peikes and Sarin 2005). 


d. Strengths and Weaknesses of the Evaluation 

One of the strengths of the SPI evaluation was its data collection strategy. The program 
office developed a rigorous data collection system that facilitated the state’s ability to provide 
common information about each participant and to track each participant’s receipt of 17 types of 
services using standard definitions and data collection procedures. This allowed VCU and MPR 
to aggregate data across projects. The projects sent their data quarterly to the SPI Project Office 


23 This discussion is based on Peikes and Sarin (2005). 
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at VCU, where it was assessed for quality and completeness, and VCU generated discrepancy 
reports. The projects then corrected the discrepancies and submitted any missing data. 

Despite the strength of the design the evaluators found the following: 1) the projects had an 
extremely difficult time collecting detailed service data from collaborating agencies and 
organizations; 2) although the projects were able to access other state administrative data sets, 
they had to overcome administrative and logistical obstacles to obtain the data; and 3) the 
projects found that tracking participants over time was much more difficult and time consuming 
than they had initially anticipated. 

The evaluators noted the following limitations in the analysis of the SSA SPI projects. 


• Insufficient Follow-Up Period. The follow-up timeframe (6 to 12 months) was too 
short to assess the effectiveness of the intervention on the goal of reducing reliance on 
public benefits. It takes a longer period of time to increase participants’ self 
sufficiency enough so that they feel comfortable leaving the SSA rolls (Kregel 2006). 

• Lack of Control Over Administrative Data. Many states used Unemployment 
Insurance data to monitor the employment and wages of SPI participants. However, 
analyses comparing these data with self-reports suggested that the Unemployment 
Insurance system did not capture some forms of employment or employers (Kregel 
2006). 

• Problems Implementing Random Assignment Study Designs. Although random 
assignment to treatment and control groups is the best way to eliminate bias in 
research, it proved to be very difficult for the projects to execute. Only four projects 
had experimental designs with some level of randomization to treatment or control 
groups. Only one (Oklahoma) randomly assigned subjects to the treatment or control 
group prior to any contact by the project. New York, New Hampshire, and Illinois 
randomly assigned volunteers to either the treatment or the control group. 

• Contamination of Control Groups. The BPAO initiative began implementation 
during the SPI project research cycle, thus providing one of the SPI services to some 
control and comparison group members. 


D. DEPARTMENT OF LABOR 
1. Customized Employment Grants 
a. Intervention and Target Population 

WIA required states and localities to bring together a range of federally funded employment 
and training services into a single system called the One-Stop Career Center system. Although 
one of the core principles of the one-stop system is to provide universal access to all job seekers, 
most had been oriented towards a self-service model that offers a “one size fits all” approach to 
customers. This resulted in people with disabilities being referred elsewhere because the One- 
Stop Career Centers were physically or programmatically inaccessible, or because staff believed 
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that other programs, such as state VR programs, could better serve people with disabilities 
(Elinson and Frey 2005). 

In three annual grant cycles between 2001 and 2003, the Office of Disability Employment 
Policy (ODEP) within DOL provided 20 grants to Workforce Investment Boards to increase the 
capacity of One-stop Career Centers to provide person-centered, individualized, or customized 
employment services for people with significant disabilities. Customized employment services 
could include strategies such as supported employment; supported entrepreneurship; 
individualized job development; job carving and restructuring; use of personal agents (including 
individuals with disabilities and family members); development of micro-boards, micro- 
enterprises, cooperatives and small businesses; and use of personal budgets and other forms of 
individualized funding that provide choice and control to the individual and promote self- 
determination [Federal Register, June 6, 2003 (volume 68, number 109)]. 

The grants funded direct customized employment services as well as the development of 
policies and practices to ensure that these strategies were systemically included in the services 
available through the one-stop system. The grants were for one year and could be renewed for up 
to four additional years. The target population was people with significant disabilities, including 
people who previously might have been considered “nonfeasible” for employment, and people 
who have been segregated in institutions, nursing homes, and day activity programs [Federal 
Register, June 6, 2003 (volume 68, number 109)]. 

ODEP also funded three other demonstration programs designed to build the capacity of the 
workforce development system to serve people with disabilities, including (1) Youth programs to 
provide services to help youth with disabilities reach their educational and occupational goals; 
(2) Technical Assistance to the youth and adult demonstration programs as well as to other 
elements of the workforce investment system; and (3) Olmsteacl WorkFORCE Action Grants. 


b. Evaluation Activities 

ODEP contracted with Westat to conduct an independent evaluation of the customized 
employment programs as part of a larger evaluation of ODEP’s grant initiatives (Elinson and 
Frey 2005). Westat collected both quantitative and qualitative data to generate their conclusions 
and recommendations. 


• Quantitative Data. Westat twice collected information on randomly selected program 
participants who had been placed in competitive employment. The first round of data 
collection included 345 customers from 25 programs. The second round included 536 
customers from 30 programs. The evaluators did not collect data on program 
participants who did not become employed, and no control or comparison groups 
were included in the evaluation design. 

• Qualitative Data. Westat conducted site visits to all ODEP demonstration locations 
and interviewed the directors of these projects, employees and partners of the 
projects, customers and their families, and community representatives. To capture 
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changes over time and collect longitudinal data, Westat researchers visited locations 
multiple times over the course of the demonstration. 


c. Findings 

The evaluation focused on two outcomes: employment and capacity building. 

Employment. Westat reported that the customized employment process can be effective in 
helping some hard-to-serve individuals with disabilities find better employment and thus allow 
them to begin exiting income support programs. This conclusion was based on the following 
findings: In the first round of data collection, 43 percent of those who self-disclosed their 
disability said that they had psychiatric or emotional disabilities, and 37 percent had never 
worked or had only worked in a noncompetitive employment environment. More than half (54 
percent) of program customers who had been placed in competitive employment obtained jobs 
with the potential for career advancement. In addition, compared to their status at the beginning 
of their involvement with the project, the percentage of customers in the ODEP demonstration 
programs who were on government benefits (such as Food Stamps, SSI, DI, and Temporary Aid 
to Needy Families) had declined in every category except subsidized housing (which increased 
as customers of the Chronic Homelessness program were placed in housing as part of their 
participation in that program). The second round of data collection on competitive employment 
placements yielded similar findings (Ellison and Frey 2005). 

Capacity Building. Demonstration programs employed multiple strategies to build capacity 
(Ellison and Frey 2005): 

• Strategic Planning. Although the programs were required to engage in strategic 
planning activities, many were still using a “plan as you go” approach. Conversely, a 
number had reinforced their understanding of local needs by gathering additional 
information from key stakeholders and involving stakeholders as part of advisory 
committees. 

• Physical and Programmatic Accessibility . Although ODEP funds could not be used 
for physical modifications, the projects served as an impetus for the sites to modify 
buildings or purchase equipment for physical and communication accessibility. Other 
approaches to enhancing accessibility at the one-stops included modifying the client 
orientation sessions to make them more disability-friendly, offering disability 
awareness training to staff, and connecting with the local Disability Program 
Navigator. 

• Training. The demonstration programs provided training (including formal training 
and technical assistance) on the following topics for their staff, partners, and 
collaborators: disability awareness, working with people with disabilities, availability 
of resources and services for people with disabilities, and advising customers on how 
to obtain financial work incentives. However, as Westat noted, the quality of this 
training varied from location to location — ranging from basic and informational to 
more thorough and problem-solving. Westat also noted that because of staff turnover 
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at the one-stop centers, it was necessary to repeat staff training to maintain 
sustainability. 

• Service and resource coordination. In all of the demonstration programs, 
coordination of services for people with disabilities (such as job accommodations, 
coaches, personal assistants, funds for training and education, and transportation) 
began with extensive collaboration among partners and other organizations. In many 
locations this coordination transitioned to the leveraging of each other’s resources; in 
others it resulted in the establishment of formal arrangements among the partners so 
that these leveraged resources could be sustained after the funding for the program 
ended. The range and types of partners varied across programs, as did the success of 
these collaborations. Despite some promising practices, the evaluation identified few 
changes to formal policies, procedures, and staff at any of the project sites, local 
Workforce Investment Boards, or collaborating one-stop centers, or changes to 
infrastructure that would facilitate systems change. 

• Sustainability. A number of elements of the demonstration program may prove to be 
sustainable, including improvements in accessibility at the one-stops, increased 
collaboration among agencies and organizations, and staff training. However, the 
ultimate measure of sustainability for the program is if customized employment 
becomes a part of the menu of services available at one-stop centers. Many project 
sites reported that it could be difficult to sustain customized employment without 
additional funding. 


d. Strengths and Weaknesses 

In the absence of information on a comparison or control group, it is not possible to gauge 
the strength of Westat’s findings or the impact of the customized employment demonstrations on 
the employment outcomes of people with disabilities. At least some of those who became 
employed after receiving services from the grant programs might have become employed in the 
absence of the grants, and perhaps all would have managed to find employment. 

2. Disability Employment Grants 24 
a. Intervention and Target Population 

In 1998, the DOL Employment and Training Administration (ETA) awarded $6.8 million in 
demonstration grants to 15 nonprofit organizations to provide employment and training services 
to persons with severe disabilities and to dislocated workers with disabilities. 25 In 2002, ETA 
awarded an additional $5.5 million to 12 organizations to pursue the same goal. 


24 This section is based on information from Almandsmith et at. (2001) except where otherwise noted. 

25 The grants ranged from ranged from $300,000 to $640,000 per program year. Grants were originally one 
year in duration, with the option of extending each over an additional two years. Although the Disability 
Employment Grant Initiative was planned as a Title IV program, in 1998 Title IV funds were supplemented with 
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The initiative gave grantees wide latitude in designing their programs, but encouraged the 
programs to invest in resources that would support participants’ efforts to develop career goals 
and skills needed to advance beyond entry-level work. The grant solicitation emphasized that 
quality of employment outcomes was more important than the number of placements. The 
project designs varied in terms of approaches, specific goals, target population, number of 
participants, funding levels, services provided, and service intensity. However, projects typically 
included recruitment, assessment, service planning, basic education, vocational training, job 
readiness training, work experience, on-the-job training, job search assistance, post-employment 
follow-along services, and supportive services. 

The target population varied across grantees and, in many cases, across sites. About half of 
the grantees concentrated on providing services to people with severe disabilities, although the 
definition of “severe” varied across sites. Some grantees focused on people with specific types of 
disabilities (physical, psychiatric, HIV/AIDS, developmental disabilities, mental retardation and 
developmental disabilities, and multiple disabilities) while others served people with any type of 
disability. Some addressed all age groups while others focused on youth transitioning from high 
school to work. Across all 15 projects receiving grants in 1998, 41 percent of participants were 
SSI/DI beneficiaries. 


b. Evaluation Activities 

In 1999, DOL contracted with Berkeley Policy Associates (BPA) to evaluate the 1998 
grantees. BPA’s evaluation of the Disability Employment Grants (Almandsmith et al. 2001) had 
four goals: 


• Assess the effectiveness of grantees in meeting their enrollment and employment 
outcome objectives 

• Analyze the grantees’ success in collaborating and coordinating with One-stop Career 
Centers and state VR agencies 

• Identify innovative practices that grantees have implemented 

• Examine the replicability of innovations, best practices, and project designs both 
within the workforce development system and for other disability employment 
service providers 


The evaluation design included two major components. The qualitative study focused on 
grant practices and collaboration strategies. It was based on site visits to the grantee’s central 
office and two of each grantee’s service sites, as well as review of grantee materials. The 


(continued) 

Title III funds to allow the demonstration to expand to include more projects. However, Title III funds were 
restricted to dislocated workers. 
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quantitative study measured participant characteristics, use of services, and employment 
outcomes. The analysis was based on reports that grantees were required to submit to DOL 
quarterly and/or annually, combined with data developed by the grantees at BPA’s request. BPA 
used univariate and multivariate techniques to explore the influence of participant characteristics, 
specific types of services, and groups of services on employment outcomes. Outcomes of interest 
included job placement rates, weeks to placement, hours per week in first job, hourly wage in 
first job, employed at 90 days, employed at 180 days, and hourly wages at 180 days. In addition, 
BPA compared demonstration outcomes to outcomes of a comparison group made up of Job 
Training Partnership Act (JTPA) program participants with substantial impairments, and who 
were served by the federal program in 1998. 26 However, BPA noted that relative to the JTPA 
comparison group, the demonstration served individuals with more severe disabilities, who were 
younger, more likely from minority backgrounds, and had more limited work histories. 


c. Findings 

In total, the Disability Employment Grants served 4,619 participants during the evaluation’s 
data collection period (July 1, 1998, to June 15, 2000). 

Employment Outcomes. Across the 15 Disability Employment Grants, 47 percent of 
participants entered employment. Thus, the program as a whole met DOL’s GPRA Program 
Year 1998 goal of a 47 percent placement rate. Across grantees, placement rates varied from a 
low of 16 percent to a high of 72 percent. As shown in Exhibit 5, the average hourly wage, 
number of hours worked, and retention rates varied substantially across grantees. 

The employment outcomes achieved by participants varied substantially depending on 
personal characteristics, the types of services individual projects offered, and the grantee’s 
approach to employment services. 


~ 6 Data for the comparison group were available from the 1998 Standardized Participant Information Report 
data which includes employment outcomes for the mainstream workforce development system, funded by the U.S. 
Department of Labor. 
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EXHIBIT 5 


DISABILITY EMPLOYMENT GRANT OUTCOMES 



Average Across All Grantees 

Minimum 

Maximum 

Percent of participants who entered 
employment 

47% 

16% 

72% 

Percent of participants who entered 
employment 20 hours or more 

42% 

10% 

64% 

Average hourly wage 

$8.01 

$5.69 

$12.62 

Average hours worked in first job 

28 hours 

16 hours 

39 hours 

Job retention rates 




90 days 

67% 

47% 

100% 

6 months 

61% 

40% 

90% 

12 months* 

55% 

37% 

100% 


Source: Almandsmith et al. (2001). 

*The 12-month results are somewhat difficult to interpret because fewer than 500 of the demonstration’s 
participants were eligible for 12-month follow-up. This is an artifact of the timing of the evaluation — data were 
collected only during the project’s first two years of operation. 

Controlling for other factors, the evaluators found the following personal characteristics to 
be associated with employment outcomes: 


• Having a history of employment before entering the grant program appears to be the 
single best predictor for success in the evaluation’s key employment outcome 
measures. 

• The higher a participant’s education level, the higher his or her hourly wage at 
placement and at six months, and the more likely the individual would be working at 
six months. 

• Participants with cognitive and psychiatric/emotional disabilities achieved 
significantly lower outcomes in terms of wages and hours per week at placement and 
retention at six months. Individuals with psychiatric disabilities were also less likely 
to be working six months after entering employment. 

• Participants with more impairments worked fewer hours per week and earned lower 
hourly wages at both placement and at six-month follow-up than did individuals with 
a single disability. 
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Analysis of the types of services provided yielded the following findings: 


• Job search services, on-the-job training and work experience services were 
significantly associated with job placement and with employment rates both 90 and 
180 days post-placement. 

> Individuals who participated in on-the-job training worked in jobs that paid 
significantly less than those held by their peers who did not use on-the-job 
training. 

> Work experience participants worked significantly fewer hours per week than did 
other participants. 


• Individuals receiving assessment and supportive services were less successful than 
participants who did not receive these services. 

> Participants who received nonvocational supports worked fewer hours per week at 
placement, and earned lower hourly wages six months after they started working, 
compared to those who did not receive such supports 27 


• Participants who received basic education services earned higher hourly wages than 
did individuals who did not use these services. 

• Post-employment services were positively associated with higher participant hourly 
wages at six months post-placement. 


Controlling for participant characteristics, the different project approaches were associated 
with the following: 


• Participants in programs using a vocational training approach had significantly better 
outcomes on three of the evaluation’s employment outcome measures. They earned 
higher wages, worked more hours per week, and were more likely to be employed at 
90-day follow-up than were participants served by projects that adopted a different 
approach to service provision. 

• Work- first projects moved participants into employment more quickly than did 
grantees that used other service approaches. Participants in these projects, however, 
earned lower hourly wages than those served by other projects both at placement and 
six months later. 


~ 7 BPA notes that this finding is consistent with grantee service designs that targeted these services to 
individuals with greater needs and barriers to employment, indicating that the analysis did not control adequately for 
variation in participant characteristics. 
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• Participants in projects that used the person-centered planning approach had poorer 
outcomes both in terms of finding a job and job retention at 90 and 180 days post- 
placement. 

• The project that used a capacity-building service strategy assisted significantly more 
participants to enter employment than did projects that used other service approaches, 
but these participants worked significantly fewer hours per week than individuals 
served by other types of projects. 


In its comparison of the Disability Employment Grant programs to the JTPA control group, 
BPA found that outcomes for demonstration participants were somewhat poorer than those 
achieved by JTPA participants. Demonstration participants had a lower placement rate, lower 
hourly wages, and worked fewer hours per week on average than did the individuals in the JTPA 
sample (Exhibit 6). 


EXHIBIT 6 

EMPLOYMENT OUTCOMES OF DISABILITY EMPLOYMENT GRANT 
DEMONSTRATION PARTICIPANTS AND JTPA COMPARISON GROUP MEMBERS 


Outcome 

Disability Employment Grant 
Demonstration Participants 

JTPA Comparison Group 

Placement 20+Hours 

41.5% 

55.2% 

Hours/Week (1st Job) 

27.8 

34.7 

Hourly Wage (1st Job) 

$7.53 

$8.11 

90-Day Retention 

67% 

68% 

12-Month Retention 

55% 

N/A 


Source: Almandsmith et al. (2001). 


Based on their understanding that the Disability Employment Grants served individuals with 
more severe disabilities than did the JTPA system at large, the BPA evaluators concluded that, 
despite the presence of participant characteristics that tended to negatively affect employment 
outcomes, once demonstration participants obtained employment, they had the same rate of 
employment 90 days later as did people with disabilities served by the mainstream workforce 
development system. Thus, they concluded that the Disability Employment Grants appeared to 
have achieved one of the purposes of the demonstration — to improve employment retention 
among the people with disabilities that they serve. 


Collaboration and Capacity Building 

ETA encouraged grantees to collaborate and develop linkages with the local workforce 
development system, the state VR agencies, and local community-based organizations providing 
employment and training services and/or serving people with disabilities. Most grantees pursued 
these goals. BPA reported that anecdotal reports from participants, project staff, and 
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collaborators indicated that grantee collaboration efforts resulted in improved outcomes and 
better coordination of services. However, neither the national evaluation nor any of the grantees 
formally measured the effects of collaboration on participant employment outcomes. Further, all 
of the grantees used more than one collaboration method in implementing their projects, thus, the 
link between specific collaboration techniques and participant employment outcomes is not easy 
to determine (Almandsmith et al. 2001). 


Innovative and Best Practices 

BPA identified the common characteristics and service practices of the grantees who 
achieved the best outcomes. They noted that the following practices seemed to be associated 
with success in one or more programs: 


• Making systematic and ongoing efforts to understand local market conditions and 
labor needs 

• Tailoring programs to individual participant skills and needs 

• Ensuring that programs are accessible and address the disability-related needs of 
participants 

• Recognizing that customer choice and customer-driven services are keys to success 

• Collaborating with formal and informal partners in the workforce development 
system in order to leverage resources and expand the number of services and 
resources available to participants 

• Creating new service partnerships 

• Collaborating with employers and the business community, particularly though a 
BAC 


d. Strengths and Weaknesses of the Evaluation 

Several aspects of the Disability Employment Grant program affected its evaluation: 


• Not all populations had access to all services and there was a close correlation 
between the population and the type of services offered, making it difficult to identify 
the effect of the intervention. 

• The project approaches varied dramatically. Although the multivariate analysis 
attempted to exploit these differences, the depth of the variation could not be 
adequately captured in the analysis. The service approaches were categorized into 
four types, but within each type, there were notable differences in the services 
offered. For example, among grantees that used the work-first strategy, one worked 
with employers to restructure jobs for participants when appropriate and feasible. 
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Others provided job coaches and other on-the-job support for participants in order to 
help employees with the training and orientation process. Another offered employers 
incentives in the form of cash grants to provide accommodations or on-the-job 
training, or a one-month trial period during which the grantee or another agency paid 
the participant’s salary. 

• For two of the types of services (on-the-job training and work experience) only one 
project provided that particular service. Thus, it is not possible to distinguish the 
effect of the particular intervention from other characteristics of the provider and 
population. 

• DOL required grantees to have sites in two or more states in order to increase the 
ability to evaluate the effectiveness of each service model in diverse settings. 
However, the grantees did not implement identical programs in each location, thereby 
hindering the ability of the evaluation to make useful comparisons between the sites. 

• As BPA noted, the time frame of the study (collecting data during the projects’ first 
two years of operation), did not provide information to assess the long-term 
effectiveness of the services provided. This is a significant limitation when 
considering the outcomes of projects with long-term goals for their participants and a 
lengthy service period. In addition, the two-year time frame yielded very small 
sample sizes for the outcome measures six and 12 months after placement 
(Almandsmith et al. 2001). 


3. Disability Program Navigators 
a. Intervention and Target Population 

DOL and SSA jointly funded Disability Program Navigators (DPNs) in 45 states, the 
District of Columbia, and Puerto Rico. The initiative establishes disability specialists in the one- 
stop centers to provide information for SSA beneficiaries and other people with disabilities 
(DOL 2007). The DPN grant is one of three funding mechanisms to support a disability 
specialist. States have also used Work Incentive Grant funding and WIA funding to support 
similar positions, often referred to as “navigator-like” staff. (Hall et al. 2005). 

The DPNs and the navigator-like staff are “systems change agents” and “facilitators.” They 
operate in the one-stop center and have some face-to-face contact with clients. However, they are 
not intended to take the role of a case manager or vocational rehabilitation counselor. Generally, 
they are tasked with the following: 


• Guide one-stop staff in helping people with disabilities access and navigate 
provisions of various programs that impact their ability to gain/retain employment 

• Provide disability-related training to one-stop staff 

• Ensure the accessibility of the one-stop center by conducting accessibility 
assessments and developing accessibility plans 
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• Reach out to the disability community to assure that clients and recipients of many 
support programs that are not partners in the one- stops become aware of, and have 
access to, the employment assistance provided through the one-stop system 

• Develop linkages and collaborate on an ongoing basis with employers to facilitate job 
placements for persons with disabilities 

• Develop partnerships with other agencies and service providers to integrate services, 
implement systemic change, and expand the capacity to serve customers with 
disabilities 


b. Evaluation Activities 

DOL contracted with the Rehabilitation Research and Training Center on Workforce 
Investment and Employment Policy for Persons with Disabilities at the Law, Health Policy, & 
Disability Center of the University of Iowa School of Law to provide the DPN project with 
training, technical assistance, and evaluation. The evaluation consists of four components 
(Schmeling and Morris 2005): 

Quarterly Report Evaluations. The evaluation instrument is not an attempt to measure 
individual outcomes, but rather an effort to understand the process of system capacity building to 
support job seekers with disabilities. It assesses the following: (1) time allocation in eight areas 
by month; (2) systems relationship activities and outcomes; (3) best practices; (4) linkages 
between one-stops and other organizations and employers; and (5) navigator needs. 

Telephone Survey of 14 States. The evaluation includes two rounds of surveys with 819 
one-stop staff and representatives from partnering agencies. The survey instrument asks about 
barriers and facilitator to services, supports, and employment outcomes for individuals with 
disabilities in the Workforce Development System, and the role of the navigator in reducing 
these barriers 

In-Depth Study of Four States. Evaluators conducted interviews with state and local 
officials in eight sites — two (one rural and one urban) in each of four states (Massachusetts, 
Llorida, Colorado, and Wisconsin). The evaluators interviewed 117 individuals to document 
implementation experiences of the DPN initiative, best practices, and short-term systems-change 
outcomes. 

Quantitative Study of Outcomes. The planned evaluation will link SSA records to one- 
stop records from four states covering the 2001-2006 period, for purposes of comparing the 
employment outcomes of people with disabilities served by one-stops in areas covered by DPNs 
to those in areas not covered by DPNs (Livermore et al. 2008). In January 2008, DOL issued a 
contract to MPR to conduct this component of the evaluation. 


c. Findings to Date and What Will Be Learned in the Future 

Findings from the quarterly reports provide quantitative information on the following: 
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• Description of typical DPN activities, including time allocation by type of activity, 
system relationships and outcomes, and involvement with organizations 

• Description of changes in DPN activities over quarters by type of activity, system 
relationships, and involvement with organizations 

• Best practice reports or “mini case studies” on systems relationships, collaborations 
with employers, and experiences of job seekers with disabilities 


Findings from the phone survey indicated that satisfaction with DPN services was high and 
that the most significant advantages of having a DPN in the system noted are (1) improving 
interagency coordination, (2) improving program/service access, and (3) improving availability 
of benefits counseling (Schmeling and Morris 2005). 

The qualitative findings of the four in-depth case studies (Emery and Bryan 2005) suggest 
that DPNs had a positive impact on one-stop centers. Respondents reported that centers were 
more accommodating and better equipped, more fully integrating services and training that are 
accessible to everyone, providing seamless service to persons with disabilities, and helping to 
realize the goal of universal access. A goal of the DPN initiative was to better enable the 
workforce network to help job seekers with disabilities secure and maintain full and meaningful 
employment. The case study report notes improved quality of referrals; less turf-protecting 
behavior and better leveraging of resources; increased awareness and more appropriate referrals 
between one- stops and community agencies; and bridging of gaps between agencies to 
coordinate services to clients. Though not a representative sample, customers interviewed during 
the site visits generally reported high levels of satisfaction with the DPNs. 

To date, the various evaluation activities have collected a large amount of descriptive 
information about the activities of DPNs, but have generated only limited analysis of data. As a 
result, the reports are most useful within the DPN project to inform other DPNs rather than to 
evaluate the impact of the project on the employment of people with disabilities. The qualitative 
findings of the case studies and phone survey are suggestive of the DPN having an impact on the 
broader service delivery system. 

If undertaken, the planned quantitative evaluation of outcomes using the linked SSA and 
one-stop data will provide useful information about the extent to which Social Security disability 
beneficiaries utilize one-stop services, the types of services they receive, and whether there have 
been changes in such utilization that might be attributed to the implementation of the DPNs. 
Estimation of impacts is likely to be problematic because DPN implementation in the states to be 
evaluated did not support the identification of a comparison. Descriptive information about 
outcomes will, however, yield substantial new information about the extent to which Social 
Security disability beneficiaries utilize the one-stop system and the characteristics and 
employment outcomes of users. To date, the quantitative evaluation of DPN outcomes has been 
delayed because of difficulties in obtaining permission to match SSA and state one-stop system 
data. 
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4. Work Incentive Grants 28 
a. Intervention and Target Population 

ETA established the Work Incentive Grant (WIG) program to address concerns about the 
ability of the one-stop system to meet the needs of people with disabilities. With four 
solicitations between 2000 and 2004, ETA awarded grants to 113 state and local entities totaling 
$65 million. The first two solicitations were relatively broad and permitted the grantees to use 
the funds for a range of activities, including assessing one-stops’ architectural accessibility; 
acquiring assistive technology; conducting outreach to the disability community; linking and 
coordinating with community disability-related agencies; training existing one-stop staff on 
disability issues; and making available staff who have the experience, knowledge, and skills 
necessary to address a broad range of disability-related issues (GAO 2004b). 

Experience with the initial grants showed that a number of local workforce investment areas 
were using disability resource specialists or navigators and that this approach seemed successful 
in improving overall service delivery for people with disabilities. As a result, in the subsequent 
rounds ETA focused its priorities more narrowly — though not exclusively — on placing a 
disability specialist in the one-stops. The grantees were also given the option to address other 
issues, such as meeting the architectural access requirements. To supplement the WIGs in 
placing staff in the one-stops, ETA, in conjunction with SSA, funded cooperative agreements to 
establish and train disability program navigators (discussed in the previous section). 


b. Evaluation Activities 

DOL contracted with the Rehabilitation Research and Training Center on Workforce 
Investment and Employment Policy for Persons with Disabilities at the Law, Health Policy & 
Disability Center at the University of Iowa to provide technical assistance to WIG grantees and 
to conduct a process evaluation. The evaluation was designed to document the progress of WIG 
activities, and identify successes, best practices, challenges, obstacles, and future needs. The 
analyses were based on annual surveys of grantees. 

The survey instrument changed slightly over the course of the evaluation, but generally 
grantees reported five types of information: (1) types of activities undertaken, (2) the amount of 
effort (none, limited, significant) and outcome (none, limited, significant) of a variety of 
activities within each of the major focus areas, (3) status of selected one-stop operations, (4) self- 
assessment of the progress made in improving access and opportunities for people with 
disabilities, and (5) major accomplishments and barriers. 


2S Unless otherwise noted, information in this section is based on several WIG evaluation reports produced by 
the Rehabilitation Research and Training Center on Workforce Investment and Employment Policy for Persons with 
Disabilities, Law, Health Policy & Disability Center, University of Iowa College of Law. These reports are available 
at http://www.doleta.gov/disability/partevidence/part4.cfm. 
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c. Findings 

WIG grantees undertook a range of activities to increase the participation and success of 
people with disabilities in the one-stop system. The activities fell into five broad categories: (1) 
accessibility, (2) identification of job seekers with hidden disabilities, (3) coordination with 
employers, (4) training and education, (5) marketing and outreach, and (6) service and 
interagency coordination. 

The evaluation focused on process rather than outcome, and did not rigorously identify the 
impacts of activities on outcomes. In each year the evaluators report the number of WIGs that 
reported “significant activity with significant outcomes” and “significant activity with limited 
outcomes” for roughly 80 activities. However they do not report any conclusions based on this 
information. 

The WIG evaluations identified the following key obstacles to success of the WIGs in 
promoting systems change: 


• State of the Economy. DOL awarded the first two rounds of grants during a period of 
state budget cuts, making it difficult for systems change programs to fully engage 
other partners and implement the proposed activities. It was also a period of high 
unemployment and the large pool of qualified job seekers added to the competition 
for limited job openings for individuals with disabilities. 

• Philosophies and Organizational Beliefs. Workforce development professionals 
tended to believe that people with disabilities required too much time and resources 
and that referring the client to other agencies (such as VR) rather than collaborating 
with VR was the most appropriate solution. 

• Performance Measures. The performance standards for the one-stop system focus on 
the percent of consumers who find jobs and increase their wages. Many stakeholders 
in the workforce system believed that individuals with multiple barriers to 
employment would be less successful in finding a job and would adversely affect the 
overall performance measures for the workforce investment area. 

• Staff Turnover. The WIGs invested in training staff about assistive technology, 
reasonable accommodations and general disability awareness, but frequent staff 
turnover at the one-stops reduced the effectiveness of this strategy. 

• Service Coordination. “Turfism” remained a systemic problem among both mandated 
and nonmandated partners at the one-stops. 

• Employer Interest and Investment. Although a majority of the WIGs implemented 
strategies to attract the interest of the business community, engaging employers was 
difficult. 
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d. Overall Strengths and Limitations 

The WIG evaluations identified the disability program navigator as a promising approach to 
addressing many of the systemic barriers faced by people with disabilities in using the one-stop 
system. This approach was replicated by other WIG grantees and prompted the introduction of 
the DPN cooperative agreements. The evaluations did not provide specific evidence of the 
impact of any particular approaches on the employment of people with disabilities. They did, 
however, identify several key obstacles to programmatic success. 


E. DEPARTMENT OF VETERANS AFFAIRS 
1. Vocational Rehabilitation and Employment Program 
a. Intervention and Target Population 29 

The purpose of the VA Vocational Rehabilitation and Employment (VR&E) program is to 
assist veterans with service-connected disabilities to prepare for, find, and maintain employment. 
The program also offers independent living services for veterans with service-connected 
disabilities that are so severe that they cannot immediately consider work. To be entitled to 
services, a veteran must have received a discharge that is other than dishonorable, have at least a 

on 

20 percent service-connected disability rating, and have an employment handicap. The basic 
period of eligibility in which VR&E services may be used is 12 years from the latter of the date 
of separation from active military service or the date the veteran was first notified by the VA of a 
service-connected disability rating. If the service-connected disability rating is less than 20 
percent or if the veteran is beyond the 12-year basic period of eligibility, then a serious 
employment handicap must be present to establish entitlement to VR&E services. Those found 
eligible may receive the following types of services: 


• Assessment to determine abilities, skills, interests, and needs 

• Vocational counseling and rehabilitation planning 

• Employment services such as job-seeking skills, resume development, and other work 
readiness assistance 

• Assistance finding and keeping a job, including the use of special employer 
incentives 

• Training such as on-the-job training, apprenticeships, and unpaid work experiences 

• Post-secondary training at a college, vocational, technical, or business school 

29 This section is based on information from http://www.vba.va.gov/bln/vre/vrs.htm . 

30 An employment handicap is defined as an impairment of a veteran’s ability to prepare for, obtain, or retain 
employment consistent with his or her abilities, aptitudes, and interests. The impairment must result in substantial 
part from a service-connected disability. 
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• Supportive rehabilitation services, including case management, counseling, and 
referral 

• Independent living services 


b. Evaluation Activities 

No formal evaluation to assess the impact of the VR&E program on the employment of 
veterans with disabilities has been conducted. However, several recent efforts have assessed the 
general performance of the program. We describe these efforts below. 


VR&E Task Force 2004 

In May 2003, the VA Secretary established a task force to conduct an independent 
evaluation and analysis of the VR&E program. The task force was asked to: 


• Conduct a functional and organizational assessment of the VR&E 

• Evaluate eligibility criteria, procedures, and processes for determining how a veteran 
is approved for training, employment, or independent living services 

• Appraise current VR&E processes, information systems, and management controls 

• Determine consistency in the administration of the VR&E Program across regional 
offices 

• Examine clinical rehabilitation practices and employment placement services used by 
other federal, state, local, or private organizations serving disabled persons, including 
veterans 


The task force based its review on a series of public fact-finding sessions, site visits, and 
analyses of previous studies and reports on the VR&E program. The fact-finding sessions 
included public meetings to solicit the comments and recommendations of Congressional 
committee staffs, GAO, veterans service organizations, Veterans Benefits Administration, and 
partnership organizations such as the Veterans Health Administration and DOL. The task force 
also received comments from VR&E staff and a variety of professional organizations and private 
sector firms prominent in the fields of disability, rehabilitation, and employment of people with 
disabilities (VA VR&E Task Force 2004). 
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PART 


In 2006, OMB conducted a PART assessment of the VR&E program. Included in the PART 

o 1 

assessment were rehabilitation rate measures. 


Office of Inspector General Audit 2007 

In 2007, the VA Office of Inspector General (OIG) performed an audit of VR&E program 
operations to evaluate program results and performance. The objectives of the audit were to (1) 
evaluate aspects of program results and performance, including accuracy of performance 
measurement and reporting; (2) assess information on the reasons veterans discontinue 
participation in the program and actions taken to reduce the probability of veterans dropping out 
of the program; and (3) evaluate and determine the effects of the statutory annual cap on veterans 
eligible for independent living benefits. The audit was based on information site visits and 
interviews with VA staff, review of documents and reports, and review of a large sample of case 
files including a comparison of the hard copy files to electronic administrative data (VA OIG 
2007). 


c. Findings 
VR&E Task Force 2004 

The general conclusions of the task force were that (1) the VR&E program has not been a 
priority of the VA; (2) the program has limited capacity to manage its growing workload; and (3) 
the VR&E program must be redesigned for the 21st century employment environment (VA 
VR&E Task Force 2004). The task force’s report included 110 specific recommendations 
covering a wide variety of management, process, and service delivery activities. As of 2007, the 
VR&E program reported that it had implemented 89 of the recommendations (VA OIG 2007). 

One of the key recommendations of the task force led to the program developing a new 
Five-Track Service Delivery System designed to make VR&E services more employment- 
focused. Veterans choose one of five tracks to achieve their employment goals, with each track 
consisting of specific services and strategies for employment and independent living that are 
most suited to the employment objectives. The five tracks include reemployment; rapid access to 
employment; self-employment; employment through long-term services; and independent living 
services (Crane et al. 2008). 


PART 

The 2006 PART assessment concluded that the VR&E program is performing adequately 
based on the annual performance goals 32 Like PART measures used in other programs, the 


31 See www.ExpectMore.gov . 

32 http://www.whitehouse.gov/omb/expectmore/summary/10003220.2006.html 
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performance measures focus on whether certain targets have been achieved, but because there is 
no counterfactual, they do not indicate the extent to which the program contributed to that 
achievement; they serve purely as descriptive performance benchmarks. Exhibit 7 shows the 
rehabilitation rate program performance measures. Although it appears that the program met its 
targets in 2007, the OIG audit report (discussed below) criticized the measures because they 
excluded participants who dropped out of the program before becoming rehabilitated. If these 
individuals had been included in the calculations, the rehabilitations rates would be much lower, 
as the majority of initial participants subsequently dropped out of the program. 

EXHIBIT 7 

VR&E PROGRAM PERFORMANCE MEASURES 


Year/Measure 

Target 

Actual 

Rehabilitation Rate 

2005 

66% 

63% 

2006 

69% 

73% 

2007 

73% 

73% 

Serious Employment Handicap Rehabilitation Rate 

2005 

N/A 

64% 

2006 

65% 

73% 

2007 

74% 

73% 


Source: www.ExpectMore.gov. 


Office of Inspector General Audit 2007 

The OIG report criticized the VR&E performance measures reported in the agency’s 
performance and accountability report (PAR). Specifically, the VR&E rehabilitation rate 
calculations were not fully explained in the PAR report. The fact that the measures excluded 
participants who discontinued participation was not disclosed. The report noted that the PAR 
should provide information on total program participants, including those who discontinued 
participation in the program, those who obtained and maintained employment, and those who 
achieved independent living goals. It also noted that most veterans discontinued participation in 
the program and were not rehabilitated, however, no information was available for the VA to 
analyze the reasons for the high rate of program discontinuation. The OIG report recommended 
that the VA better explain the methodology used for VR&E rehabilitation rate calculations 
reported in the PAR; ensure that the PAR includes data on total program participants; and 
develop methods and procedures to determine why veterans discontinue participation in the 
program before they are rehabilitated (VA OIG 2007). 
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F. INTERNAL REVENUE SERVICE 


1. Business Tax Incentives to Hire and Accommodate People with Disabilities 
a. Intervention and Target Population 

Three federal tax incentives were created to help businesses with the costs of making their 
facilities accessible, and to encourage the employment of people with disabilities. 

The Work Opportunity Tax Credit (WOTC), established under the Small Business Job 
Protection Act of 1996, applies to many economically disadvantaged workers, including those 
with disabilities using veterans, vocational rehabilitation, or Ticket to Work program services, or 
receiving SSI. 3 ' 1 The WOTC gives employers a tax credit when they hire or retain economically 
disadvantaged individuals. A firm hiring a qualified individual who remains employed for 400 or 
more hours during the first 12 months can receive a credit of 40 percent of the individual’s 
qualified wages up to a maximum on $6,000. If the individual is employed for at least 120 hours, 
but fewer than 400 hours during the first 12 months, the business can receive a tax credit of 25 
percent of the individual’s qualified wages, again to a maximum of $6,000. 14 

The Disabled Access Tax Credit (Section 44 of the IRS Code), introduced in 1990, gives a 
tax credit of up to $5,000 to small businesses for certain costs they incur when making their 
workplace ADA compliant. This includes costs incurred to: 


• Remove architectural, communication, physical, or transportation barriers that 
prevent a business from being accessible to, or usable by, individuals with disabilities 

• Provide qualified readers, taped texts, and other effective methods of making 
materials accessible to people with visual impairments 

• Provide qualified interpreters or other effective methods of making orally delivered 
materials available to individuals with hearing impairments 

• Acquire or modify equipment or devices for individuals with disabilities 

• Provide other similar services, modifications, materials, or equipment 


The Barrier Removal Tax Deduction (Section 190 of the IRS Code), introduced in 1976 
(and amended in 1990), allows a deduction of up to $15,000 per year for capital expenses 
incurred when a business of any size makes its building or transportation more accessible for 


33 See IRS Form 8850 and instructions for definitions for the groups covered by the WOTC. 

34 Different limits and time periods apply if the employee is a member of the “Long-term Family Assistance 
Recipient” or “Summer Youth” categories of the WOTC. 

35 See IRS Form 8826. 
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people with disabilities. This deduction can be used in combination with the Disabled Access 
Credit if the expenses qualify under both provisions. 36 


b. Evaluation Activities 

In 2002, GAO published an evaluation of the effectiveness of the three business tax 
incentives (GAO 2002c). The evaluation specifically focused on the effectiveness of the 
provisions in promoting the employment of people with disabilities. GAO analyzed tax data from 
the Internal Revenue Service (IRS) Statistics of Income Programs for 1999. However, the 
findings of that component of the study were extremely limited for a number of reasons. First, 
there was no information on the barrier-removal deduction. For the other incentives, only the 
credit amount was listed, so it could have applied to accommodations not related to the 
employment of people with disabilities; the WOTC can apply to other groups — not just people 
with disabilities — and the Disabled Access Credit can be used for improvements in customer 
accessibility. Additionally, the tax return data were not adequate for assessing how the tax 
incentives affected employer attitudes toward hiring, retaining, and making accommodations for 
people with disabilities. 

To complement the very limited data available from the IRS, the GAO performed a 
literature review and conducted interviews with many people with interests relevant to the tax 
incentives. In its literature review, GAO was only able to find studies pertaining to the WOTC. 
Interviewees included officials from the Departments of Education, Labor, Justice, and the 
Treasury, and the Equal Employment Opportunity Commission; state officials in New York and 
California; businesses of various sizes; tax preparers; disability organizations; academic experts; 
and business groups. 


c. Findings 

Based on the analyses of IRS data for 1999, GAO found that only a very small share of 
corporations or individuals with business affiliations (for example, sole proprietorships, 
partnerships, farms, and individuals with rental properties) used the two business tax provisions 
for which IRS data were available: 


• One out of every 790 corporations and one out of 3,450 individuals with a business 
affiliation reported the WOTC on their tax returns. Corporations in the retail and 
service industries accounted for the largest share of the credits. The total value of the 
credits was approximately $254 million, with the average credit being about $106,000 
among corporations and $3,800 among individuals with a business affiliation 
reporting the credit on their tax returns. 

• One out of every 680 corporations and one out of 1,570 individuals with a business 
affiliation reported the Disabled Access Credit on their tax returns. Individual 


36 See IRS Publication 535, Business Expenses. 
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taxpayers with a business affiliation accounted for the majority of the credits. 
Individual and corporate taxpayers associated with the providers of health care and 
other social assistance services accounted for the largest share of the disabled access 
credits. The total value of the credits was approximately $59 million, with an average 
credit of about $2,800. 


Through interviews and literature reviews, GAO found some indication that employers do 
take tax incentives into consideration, but could not determine if the incentives directly helped 
people with disabilities find or retain employment. Additionally, it was unclear if the credits 
given to employers were for people they would have hired anyway, without the credit. 

Interviewees suggested ways for improving the effectiveness of the tax credits: increasing 
outreach so more people would be aware of the incentives; making the requirements clearer; 
raising the maximum claim amount; and expanding eligibility. But GAO was unsure if the 
suggested changes would lead to increased effectiveness of the provisions at an acceptable cost 
to the government. 

GAO’s evaluation of the effectiveness of the business tax incentives was severely hampered 
by a lack of data and an inability to determine the effects on employment given the data available 
and the manner in which the provisions were implemented. The study was unable to definitively 
determine whether the incentives had any real effect on the hiring or retention of people with 
disabilities. 


G. SOCIAL SECURITY ADMINISTRATION 
1. Accelerated Benefits Demonstration 
a. Intervention and Target Population 

In January 2006, SSA awarded a contract to MDRC to conduct the Accelerated Benefits 
(AB) Demonstration. The AB demonstration is a study designed to assess whether providing new 
DI beneficiaries who are uninsured with health insurance benefits and other supports will 
stabilize or improve their health and help them return to work. In this demonstration, newly 
entitled DI beneficiaries (that is, those who have at least 18 months of the 24-month waiting 
period for Medicare eligibility remaining) ages 18 to 54 and who have no health insurance 
coverage were randomly assigned to three groups: a control group that just received their regular 
DI benefits; a treatment group that received immediate access to health care benefits (AB Basic); 
and a second treatment group that received health care benefits and additional care management 
and employment-related supports (AB Plus). The study will assess whether the provision of AB 
and AB Plus for a period of two to three years will help new beneficiaries improve their health 
and functioning and return to work. The study will be conducted over a five and a half-year 
period. 
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b. Evaluation Activities 


The AB demonstration is using an experimental design to evaluate how the intervention 
affects the health, health care utilization, employment, and DI benefit receipt of a sample of new 
DI beneficiaries who are randomly assigned to one of two treatment groups (AB Basic and AB 
Plus). A total of 2,000 DI beneficiaries residing in 53 large metropolitan areas will participate in 
the study (Wittenburg et al. 2008). While the demonstration features and evaluation design are 
not fully specified in public, based on the RFP issued by SSA and early information provided 
by MDRC, 38 it appears that the evaluation will include process, impact, and cost-benefit analysis 
components. Some of the key research issues to be addressed by the evaluation include the 
following: 


• What are the challenges faced in creating and implementing a system of medical 
benefits? 

• How does accelerated access to health benefits affect health care service utilization, 
health outcomes, employment, and reliance on DI benefits? 

• What is the added effect of providing care management in addition to health benefits? 

• Does accelerated access to health benefits have larger effects for some groups of DI 
beneficiaries than for others? 


In addition, SSA has requested that a cost-benefit analysis be conducted in order to 
understand the annual cost of the project and whether the program’s cost can ultimately be 
justified by the long-term benefits to the beneficiaries and by potential savings to the Trust Funds 
and other federal programs. 

The evaluation will rely on data obtained from SSA administrative files, baseline and 
follow-up surveys of study participants, demonstration operations data, and site visits and 
interviews with intervention sites. 


c. Progress to Date and What Will Be Learned in the Future 

The initial phase of the demonstration began in November 2007 and full rollout began in 
March 2008. Evaluation reports are expected to be completed in February 2009, October 2009, 
and January 2011 (Wittenburg et al. 2008). 

Based on the information currently available, it appears that the impact evaluation will 
provide substantial information on the short- and longer-term impacts of providing public health 
insurance to DI beneficiaries earlier than under current law. The evaluation will focus on the 


37 SSA-RFP-05-1045 issued July 19, 2005. 

38 MDRC (2006) and Wittenburg et al. (2008). 
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impact of the two variations of the intervention on health status, health care utilization, 
employment, earnings, duration of employment, and dependence on DI benefits. The findings 
from the evaluation should also be able to provide information about the determinants of 
successful return to work and the beneficiary characteristics most correlated with successful 
outcomes. 

Based on the RFP issued by SSA, it appears that the cost-benefit analysis will focus on 
estimating the annual costs (including net cost) of the demonstration project and the annual cost 
(including net cost) that would have been incurred in the absence of the project. The evaluation 
will consider the costs and savings to the Trust Funds and other federal programs including long- 
term costs and savings. It does not appear that costs or benefits occurring outside of federal 
programs (for example, state expenditures on Medicaid and other assistance programs, or costs 
to employers) will be considered. 

It does not appear that the evaluation will assess whether the provision of immediate health 
benefits leads to any induced demand for DI benefits. As the demonstration is targeting only 
uninsured DI beneficiaries, is also unlikely that the evaluation will assess the degree to which 
accelerated access to health benefits crowds out other sources of insurance (such as employer- 
sponsored insurance or Medicaid). 


2. Benefit Offset Demonstration 
a. Intervention and Target Population 

The Ticket Act requires SSA to conduct demonstration projects for the purpose of 
evaluating a program for DI beneficiaries under which benefits are reduced by $1 for each $2 of 
the beneficiary’s earnings above a specified amount. The Benefit Offset National Demonstration 
(BOND) will test a $1 reduction in benefits for every $2 in earnings above SGA, with the goal of 
enabling more beneficiaries to return to work and maximize their employment, earnings, and 
independence. Currently, DI beneficiaries lose their entire cash benefit if their earnings exceed 
the Substantial Gainful Activity (SGA) threshold by any amount for a sustained period. BOND 
will also test the provision of intensive benefits counseling. Some treatment groups will be 
provided with more intensive versions of currently available benefit counseling services, 
allowing more personalized support to help with the complications of returning to work. Some 
evidence suggests that financial incentives have a limited impact on employment and earnings 
because participants lack understanding of the benefit schedules; provision of more 
comprehensive benefits counseling might therefore increase the impact of the incentives. The 
more intensive counseling will be tested both in combination with the offset and as the sole 
addition to the current DI work incentives. The offset component of the treatment will be offered 
to samples of Dl-only and concurrent beneficiaries residing in 10 sites across the country. The 
intensive benefits counseling component will be offered only to a sample of Dl-only 
beneficiaries in these sites. The contract for the design of the national study was awarded to Abt 
Associates on September 30, 2004 (SSA 2007). 

Concurrent with the design of the national demonstration, SSA sponsored a four-state pilot 
offset demonstration in Connecticut, Utah, Vermont, and Wisconsin. The purpose of the four- 
state pilot was to collect early information on the benefit offset demonstration that will be useful 
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in developing the national study. The four-state pilot cooperative agreements were awarded in 
April 2005, and the states began recruiting participants for the demonstration in August 
2005. 39 Treatment group participants in the state pilots receive a $1 for $2 DI benefit offset for 
earnings above SGA for up to 72 months following the usual nine-month DI trial work period. 
Each state in the pilot instituted its own methods for participant recruitment, informed consent, 
and the nature of any complementary intervention services that would be provided to pilot 
participants (for example, benefits counseling or referrals to local employment supports). The 
four-state pilot will terminate in January 2009. 40 


b. Evaluation Activities 

Details of the demonstration and evaluation designs for BOND are not yet public. Based on 
information contained in the original RFP, it appears that BOND will be conducted using a 
random assignment design at a set of sites selected such that they are nationally representative of 
all DI beneficiaries in the BOND target population. Large sample sizes will be needed to detect 
fairly small impacts of the interventions on employment and benefit receipt outcomes. The 
evaluation will likely include impact, participation, process, and cost-benefit analysis 
components and rely on data collected from a variety of sources, including SSA and CMS 
administrative data, process data generated by the intervention, beneficiary surveys, and site 
visits and interviews with demonstration staff and other community stakeholders. The evaluation 
will focus on outcomes related to participation, employment, earnings, SSA benefit receipt, 
income, and reliance on other public assistance programs. 

Each state involved in the four-state pilot designed its own evaluation and data collection 
methods. All were required to implement a random assignment design, but otherwise had a large 
degree of flexibility in how to implement the intervention, collect data, and conduct evaluations 
of the implementation process and demonstration outcomes. Project enrollment information, 
state program administrative data files (including Medicaid, unemployment insurance, and 
vocational rehabilitation), and interviews with participants were the primary sources of data to be 
used to assess the pilot projects. 


c. Progress to Date and What Will Be Learned in the Future 

BOND was initially scheduled to begin in late 2007, 41 but design changes have delayed the 
start of the demonstration until a currently unknown date. As the demonstration will use an 
experimental research design, the evaluation is expected to produce rigorous estimates of the 
impact of the offset and intensive benefits counseling on employment and benefit receipt 
outcomes. 


39 http://www.socialsecurity.gov/disabilityresearch/offsetpilot2.htm 

40 Federal Register , Volume 73, Number 239, pp. 75492-75494, December 11, 2008. 

41 http://www.socialsecurity.gov/disabilityresearch/offsetnational2.htm 
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As noted above, the four-state pilot has been underway since late 2005. Each of the four 
states presented preliminary evaluation findings at a March 2007 meeting sponsored by SSA. 
Some of the general findings from the states’ experiences include the following (Silverstein and 
Jensen 2007): 


• Each of the states experienced difficulty recruiting and enrolling large numbers of 
participants. Although the projects found it useful to use existing disability service 
providers (such as VR, mental health, and development disabilities providers) to 
recruit participants, many candidates were not eligible for the demonstration. The 72- 
month extended period of eligibility limit in the demonstration served to exclude 
many DI beneficiaries who had demonstrated a capacity to work from participating in 
the project. In addition, the informed consent process was very time intensive. 

• The role of benefit counseling was critical in the demonstration. The complexity of 
the offset, the fact that benefit adjustments were not made in a timely manner by 
SSA, and information about how and when to report earnings were all factors that 
frequently needed to be addressed by benefits counselors. 

• All states experienced delays with respect to SSA conducting work activity reviews 
and accurate benefit adjustments. 


States had a number of suggestions for making BOND as effective as possible. Primary 
among them was the need to have sufficient benefits counseling resources available for 
participants, to try to keep the intervention as simple as possible for beneficiaries to understand, 
to have the mechanisms in place for SSA to provide timely and accurate information about 
benefits and eligibility status, and to make benefit adjustments in response to earnings changes in 
a timely and accurate manner so that over- and underpayments are avoided. More detailed 
participation and outcome information may be forthcoming from the state pilots, which will 
terminate in January 2009. 


3. Benefits Planning, Assistance and Outreach/Work Incentive Planning and Assistance 
a. Intervention and Target Population 

SSA, as authorized by Section 121 of the Ticket Act, awarded 116 cooperative agreements 
nationwide to a variety of community organizations under the auspices of the Benefits Planning, 
Assistance and Outreach (BPAO) program beginning in 2000. 42 The purpose of the BPAO 
initiative is to provide SSA disability beneficiaries (both SSI and DI beneficiaries) with accurate 
and timely information about SSA work incentives and other federal efforts to remove regulatory 
and programmatic barriers to employment for persons with disabilities. Trained benefits 


42 In the initial competition for FY 2000, 43 BPAO projects were funded for the first five-year cycle ending on 
September 30, 2005. Fifty-one projects were funded in the second cycle, which began on lanuary 1, 2001, and ended 
on December 31, 2005. The third cycle of 22 projects began on luly 1, 2001, and ended lune 30, 2006. 
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specialists in local BPAO programs work with individual beneficiaries to explain the many 
regulations, provisions, work incentives, and special programs that complicate an individual’s 
decision to enter or reenter the workforce. Benefits specialists are instructed not to tell 
beneficiaries what to do or to make specific recommendations. Instead, they are to allow 
beneficiaries to make their own informed decisions based on complete and accurate information. 
In addition, they support individuals who choose to enter employment by assisting them to 
comply with all relevant regulations and reporting procedures. The primary purpose of the 
programs is to educate beneficiaries about work incentives. Although increased employment 
would be a desirable outcome for the BPAO program, the impact of the initiative on employment 
is a secondary consideration and need not be positive. 

Each BPAO project conducts an outreach program in its area to identify beneficiaries who 
might be eligible for their services. SSA does not provide any lists regarding current 
beneficiaries but may make referrals to a local BPAO project. The beneficiaries are given 
counseling by benefits specialists during one or more contacts. These contacts may take place on 
the telephone or in person. The effect of their work activity is discussed, and the ramifications of 
this work for their benefits are explained. Benefits specialists may contact third parties on behalf 
of beneficiaries to verify information or to assist beneficiaries in carrying out the actions required 
of them. As of September 2006, over 250,000 beneficiaries had received assistance from one of 
the BPAO organizations operating nationwide 43 

Benefit specialists under the BPAO program received two weeks of intensive training on 
work incentive issues. The training is provided by one of three BPAO technical assistance 
centers funded by SSA. 44 After the initial training, benefit specialists receive ongoing technical 
assistance and support through these centers. 

In 2006, SSA renamed the BPAO initiative the Work Incentive Planning and Assistance 
(WIPA) project and issued a new Request for Applications for WIPA projects. Benefit specialists 
under the WIPA program are referred to as Community Work Incentives Coordinators (CWICs). 
The WIPA projects operate similarly to the BPAO programs, but will emphasize improved 
community partnerships. One of the ways in which this will be done is by conducting periodic 
Work Incentives Seminars (WISE) to provide beneficiaries the opportunity to meet directly with 
WIPAs, Employment Networks, and public and private community-based organizations that 
provide services to people with disabilities. Each WISE is expected to last approximately two 
hours and will provide beneficiaries with information about available work incentives and job 
supports needed to either assign their Ticket or pursue other employment options. SSA made the 
first round of WIPA awards in September 2006, with awards to 99 projects in 49 states. A 
second-round application process was implemented in late 2006 to recruit additional programs 
for uncovered areas of the country. As of mid-2007, 104 WIPA projects were operating 
nationwide. Most had previously operated as a BPAO program, but a number of new 


43 http://www.vcu-barc.org/NatReport/raw060930.html 

44 The three organizations awarded contracts by SSA to conduct program training and provide technical 
assistance to BPAOs were Virginia Commonwealth University, Cornell University, and the University of Missouri. 
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organizations have become involved as WIPAs.45' 46' 47 In August 2007, SSA awarded a 
contract to Virginia Commonwealth University to act as the training and technical assistance 
center for the WIPA projects. 


b. Evaluation Activities 

We have identified two efforts to evaluate aspects of the BPAO programs, and a third 
evaluation effort for the WIPA projects that is currently in the planning stage. 


Evaluation of Vermont’s Benefits Counseling Program (Tremblay et al. 2004) 

Vermont established a specialized benefits counseling program in 1999 under its SSA- 
sponsored State Partnership Initiative. Although not technically a BPAO program when it began 
in 1999, it operated in much the same way and subsequently obtained funding from SSA to 
operate as a BPAO. 

Staff of the Vermont Division of Vocational Rehabilitation conducted an evaluation of the 
impact of benefits counseling on a sample of 672 SSA disability beneficiaries who were state VR 
service users and who had received benefits counseling services between 1999 and 2002. The 
study used a quasi-experimental design to estimate the impact of the benefits counseling on 
earnings. The earnings of treatment group members over a four- year period (two years before the 
intervention and two years after) were compared to the earnings of members of two matched 
comparison groups who did not receive benefits counseling services: a “contemporaneous” 
comparison group comprised of VR service users who received services at the same time as 
members of the treatment group; and an “historical” comparison group made up of VR service 
users who had received services approximately two and a half years earlier than the treatment 
group members. The historical comparison group was used to confirm findings from the 
contemporaneous group, as the latter was susceptible to treatment spillover effects. Both 
comparison groups were matched to treatment group members based on a variety of VR and 
SSA program-related characteristics. 

The analysis used quarterly earnings data on treatment and comparison group members over 
a four- year period obtained from Vermont’s Unemployment Insurance program. Regression 
modeling was used to analyze differences in the longitudinal earnings patterns between treatment 
and comparison group members. 


45 http://www.ssa.gov/work/ServiceProviders/wipafactsheet.html 
4,1 http://www.ssa.gov/work/WIP ARP 1 A_FAQ.html 

47 http://www.socialsecurity.gov/work/ServiceProviders/WIPADirectory.html 
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2002 BPAO Satisfaction Survey (SSA 2004) 


Between November and December 2002, SSA conducted a phone survey with 1,764 
individuals who were identified as having received services from one of the 116 operating BPAO 
programs between October 2001 and August 2002. The purpose of the survey was to obtain 
feedback about the experiences and opinions of BPAO program participants. The study was not 
intended to assess impacts of the program on beneficiary knowledge, behavior, or employment. 

Using SSA’s standard six-point rating scale that ranges from “excellent” to “very poor,” 
beneficiaries were asked to evaluate their overall satisfaction with the information and services 
provided, as well as the more tangible aspects of the experience such as the convenience, 
accessibility, and privacy of the location where they met with the benefits specialists. 
Beneficiaries also provided their level of satisfaction (“very satisfied” to “very dissatisfied”) with 
access to the benefits specialist, both in person and by telephone. The clarity and helpfulness of 
the information and actions of the benefits specialists were assessed using a four-point scale from 
“strongly agree” to “strongly disagree.” The survey also contained questions about the 
participant’s work activity and education. 


SSA’s WIPA Evaluation 

In 2007, SSA awarded a contract to MPR to conduct a process evaluation of the WIPA 
program. The evaluation will focus on the initial stages of the WIPA program, including the 
transition out of the BPAO program. The process analysis is intended to serve as a foremnner to 
an outcomes-oriented evaluation that might be undertaken in the future, and as a means to 
capture any data that might be lost after the early phase of WIPA implementation is complete 
(Buschman et al. 2007). 

The process evaluation will document how the WIPA program was implemented, how the 
program is monitored by SSA, the characteristics of the organizations acting as WIPAs and the 
services they provide, the characteristics of the beneficiaries served by WIPAs, interactions and 
competition with other local organizations, the kinds of data that are collected and maintained by 
the WIPAs, and stakeholder views about what constitutes program success. The evaluation will 
be based on interviews, site visits, and focus groups conducted with SSA staff, WIPA staff, and 
beneficiaries served by the program. It will also include a review of documents, files, and data 
maintained by the WIPA programs (Buschman et al. 2007). 


c. Findings and What Will Be Learned in the Future 

From the Tremblay et al. (2004) study, there is some evidence that benefits counseling had a 
positive impact on the earnings of SSA disability beneficiaries in Vermont. Although there are a 
number of limitations associated with the study in attempting to draw conclusions about the 
effectiveness of BPAO programs in general (comparability of the treatment and comparison 
groups, potential effects of selection bias, comparability of the intervention and service 
population to that of other BPAO programs), the findings of a significant positive impact on 
employment cannot be completely discounted. 
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We also know from the SSA (2004) satisfaction study that, in general, beneficiaries 
receiving benefits counseling services from BPAOs were satisfied with the services they 
received (89 percent rated the services as excellent, very good, or good). While most 
beneficiaries were generally satisfied overall and with various aspects of their services, a 
minority expressed complaints related to the inability of benefits specialists to provide complete 
and accurate information, failure to return calls promptly, and lack of tangible help in finding 
employment. The convenience of the meeting location was one of the lowest-rated aspects of 
service (84 percent rated it excellent, very good, or good) because of the long distance 
respondents had to travel, the lack of transportation, and inadequate parking. Respondents also 
reported the lowest level of agreement (87 percent) with the statement that, after talking with the 
benefits specialist, they understood what they were supposed to do or what was supposed to 
happen next. 

Findings from the WIPA process evaluation are expected to be released in early 2009. From 
this evaluation, we will gain a better understanding of how the benefit specialists are operating, 
stakeholder perceptions of how well the program is performing, and the potential for an 
assessment of program outcomes. Given the manner in which the WIPA programs are being 
implemented, it is unlikely that a rigorous impact analysis of the effects of the program on 
employment could be conducted on a national scale. Smaller-scale assessments, however, might 
be possible with the cooperation of specific WIPA programs. While much could be learned from 
a small-scale, rigorous assessment of the impact of WIPA programs on employment, given the 
potentially large degree of variation across programs (for example, in service quality and focus, 
characteristics of clients, and local factors affecting program performance), it would be difficult 
to extrapolate the findings to the national program. A broader but less rigorous investigation of 
employment outcomes, combined with the information of the nature that is planned for the 
WIPA process evaluation, might be useful in identifying the factors that appear to be associated 
with improved employment outcomes. 


4. Employment Support Representatives/Area Work Incentive Coordinators 
a. Intervention and Target Population 

Section 121 of the Ticket to Work and Work Incentive Improvement Act of 1999 mandates 
that SSA shall “. . . establish a corps of trained, accessible, and responsive work incentives 
specialists . . .” to assist beneficiaries with disabilities who want to start or continue working. 
The mandate came in response to criticisms of SSA by beneficiaries with disabilities who want 
to start or continue working, their families, advocates, and service providers, who complained 
that SSA field staff lacked adequate capability to provide accurate and accessible information 
about SSA’s employment support programs and to process disability work-issue workloads 
timely and accurately. 

In response to the Ticket Act mandate, SSA began piloting a new, temporary position, the 
Employment Support Representative (ESR), with 32 ESRs serving 54 sites nationally. SSA 
piloted the ESR position by testing three different models to determine whether and how best to 
implement the position nationally. The ESR pilot began in July 2000 and concluded in 
September 2001. ESRs were selected from among claims representatives, underwent six weeks 
of intensive training on SSA work incentive provisions and related issues, and received a 
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promotion to the ESR position. In addition to the role of providing beneficiaries information 
about work incentive provisions, the 32 ESRs were tasked with conducting outreach and 
providing information to the general disability community. The ESR pilot ended in September 
2001, and SSA issued an evaluation of the pilot in November 2001 (SSA 2001). 

Based in part on the findings of the ESR pilot, in May 2003 SSA implemented two new 
types of staff positions: an area work incentive coordinator (AWIC) and a work incentives 
liaison (WIL). AWICs are responsible for providing assistance to SSA field office staff on 
employment support and outreach issues by coordinating and/or conducting local public outreach 
on work incentives; providing, coordinating, and/or overseeing training for all personnel on 
SSA’s employment support programs; handling sensitive or high profile disability work- issue 
cases; and monitoring the disability work-related issues in their respective areas. The 58 AWICs 
coordinate with the 1,335 WILs in local offices, the public affairs specialists, the Plan for 
Achieving Self-Support Cadre members, and other SSA personnel to provide improved services 
and information on SSA’s employment support programs. Each AWIC provides technical 
support and training to 20 to 30 field offices and networks with community agencies and other 
organizations that provide employment services to people with disabilities. The WILs provide 
technical assistance to staff and field office management, including assistance on complex cases. 
The WILs also process cases involving work incentives and maintain ongoing contact with 
beneficiaries. WILs are also involved in TTW marketing and outreach, but their level of 
involvement varies according to the needs of the SSA field office. 48 


b. Evaluation Activities 

As noted above, SSA began a pilot of the ESR position in July 2000, and completed an 
internal evaluation of the pilot in November 2001 (SSA 2001). The pilot tested three variants of 
an ESR model: 


• Model 1, Expanded Field Office (FO) Model: ESRs were stationed in an SSA FO and 
served that FO and, in some instances, affiliated FOs. The ESRs were expert in all of 
SSA’s employment support programs, and adjudicated Plans for Achieving Self 
Support (PASSs). 

• Model 2, Expanded Cadre Model: ESRs were stationed in an existing PASS Cadre. 
They served a specific FO service area(s) within the PASS Cadre service area. These 
ESRs needed to travel to the FOs that they served. They were expert in all of SSA’s 
employment support programs, and adjudicated PASSs. 

• Model 3, Field Office/Cadre Model: ESRs were stationed in an SSA FO and served 
that FO and, in some instances, affiliated FOs. The ESRs were expert in all of SSA’s 
employment support programs, but unlike under the other models, these ESRs did not 


48 Based on information from Thornton et at. (2004) and from http://www.socialsecurity.gov/ 
work/Beneficiaries/awic.html. 
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adjudicate PASSs. The PASS Cadre continued to be responsible for the adjudication 
ofPASSs. 


The evaluation of the pilot used data from several sources to assess the impact of the new 
position on service delivery, productivity, and customer satisfaction. The impact on beneficiary 
employment was not in the scope of the evaluation. The sources of data used included: 


• Surveys sent to five groups affected by ESRs: customers and community-based 
organizations that received ESR services; ESRs; ESR pilot site managers and area or 
state directors; and other SSA field employees at ESR sites 

• Information collected during a teleconference conducted with all ESRs in August 
2001 

• ESR work activity as captured on an “electronic work ticket” and through time and 
attendance data 

• ESR outreach costs 


c. Findings 

The SSA (2001) ESR evaluators concluded that the position was a success. The feedback 
from the beneficiaries and staff of community organizations who were surveyed indicated 
overwhelming appreciation for and satisfaction with the single point of contact and the 
responsiveness of ESRs. The location of the ESR was viewed as crucial to the success of the 
position in dealing with customers, community organizations, and fellow employees. Models 1 
and 3, where ESRs were located in and dedicated to a specific FO, performed notably better in 
terms of work processing, customer satisfaction, and response time. 

Work- year projections indicated that, when all types of employment support activities (that 
is, work CDRs, SGA determinations, employment support inquiries, and post-entitlement 
actions) were considered, Model 1 was the most productive. However, Model 3 produced the 
highest number of employment support activities in the shortest average time. 

In light of the success of the pilot, the evaluation recommended that SSA make the ESR a 
permanent position and ensure that as many service areas as feasible have ESR services. Due to 
cost and resource considerations, however, SSA decided to implement the ESR concept in the 
form of the AWIC/WIL model described previously, a more diluted version of the ESR. AWICs 
have a high skill level and expertise related to work and post-entitlement issues generally 
equivalent to that of the ESRs, but act more as technical assistance resources and trainers for the 
WIL staff located in the FOs — who in turn deal directly with beneficiaries but generally have 
less expertise in work-related issues than the AWICs. 
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5. Florida Freedom Initiative 49 
a. Intervention and Target Population 

The Florida Freedom Initiative (FFI) is a demonstration designed to test whether the 
provision of employment and microenterprise services, along with certain SSI waivers, will lead 
to greater employment, assets, and self-sufficiency among Social Security disability beneficiaries 
participating in the program. The target population is adult SSI recipients with developmental 
disabilities who participated in Florida’s Cash & Counseling program, called Consumer Directed 
Care (CDC)-plus. CDC-plus provides people receiving personal assistance services an allowance 
to hire personal care workers and to buy other disability-related supports in lieu of the Medicaid 
home- and community-based benefits they would otherwise have received. 

Under FFI, volunteer participants receive information and training about employment and 
microenterprise development, and assistance with addressing human capital barriers to 
employment. In addition, FFI participants are subject to several waivers of SSI regulations, 
which are intended to promote employment and encourage asset accumulation. These waivers 
include the following: 

1. Exclusion of Medicaid Payments being saved for the purchase of medical or 
social services from the participants’ countable income in determining SSI 
payments and eligibility. This waiver is required to allow SSI beneficiaries to 
keep SSI eligibility while participating in the CDC-plus program (which provides 
Medicaid beneficiaries with a cash allowance so that they can manage their own 
care instead of receiving Medicaid’s home and community-based waiver 
services). 

2. Expansion of Individual Development Accounts (IDAs). The FFI IDAs are 
savings accounts excluded from SSI resource limits when determining SSI 
eligibility. Although some IDAs are set up to contribute federal or state matching 
funds to reward participants for saving, the IDAs offered under the FFI program 
did not have this feature (as of program implementation). 

3. Increased Exclusion for Earned Income. Currently, the monthly benefit checks 
of SSI recipients who work are reduced by 50 cents for every dollar earned after 
a $20 income exclusion and a $65 earned-income exclusion. This waiver raises 
the earned-income exclusion from $65 to $280 per month, in effect allowing 
those who earn more than $280 to keep about $107.5 more of their earnings each 
month. 

4. Modified Goal for a Plan for Achieving Self-Support (PASS). A PASS allows a 
beneficiary to set aside income or resources for a specified time to save for a 
work goal. Under the usual PASS rules, someone who establishes a PASS for 
education must specify the work goal. This waiver allows FFI participants to 


49 Unless otherwise noted, the information in this section is based on Peikes and Dale (2005). 
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specify post-secondary education as a PASS goal without initially specifying the 
work goal, as long as the PASS includes a step for specifying a work goal at least 
one year before completing the coursework. 

5. Suspension of Continuing Disability Reviews (CDRs). SSA must periodically 
review medical and other evidence to determine whether an individual continues 
to meet the eligibility requirements for benefits. This waiver suspends CDRs for 
FFI participants while they participate in the demonstration. 

The FFI demonstration was collaboratively sponsored by federal and state agencies, 
including Florida Agency for Persons with Disabilities; the Office of the Assistant Secretary for 
Planning and Evaluation in the U.S. Department of Health and Human Services; SSA; and the 
CMS. The FFI program was funded from September 2003 through September 2007. FFI enrolled 
its first participant on September 30, 2005. 


b. Evaluation Activities 

The Office of the Assistant Secretary for Planning and Evaluation in the U.S. Department of 
Health and Human Services issued a contract to MPR to design an evaluation of the FFI 
demonstration. The proposed design (completed in May 2005) involved an analysis of the FFI 
program implementation, a participation analysis, and an analysis of employment, earnings, asset 
accumulation, and benefit receipt outcomes based on a pre-post comparison of outcomes among 
FFI participants. The impact evaluation was designed to be based on SSA administrative data 
and on data collected via baseline and follow-up surveys of FFI participants. 

The proposed pre-post design for estimating the impact of FFI on employment, earnings, 
assets, and benefits was arrived at based on several considerations: due to low initial enrollment 
and the small target population from which participants would be recruited (700 CDC-plus 
participants), it was believed that sample sizes would be too small to support a more rigorous, 
random assignment design; and because the target population had just recently undergone a 
random assignment process (to become participants in the CDC-plus program), FFI program 
officials did not want to adopt a randomized design for FFI. Non-experimental designs were also 
considered, but rejected due to concerns about selection bias, low enrollment, and the costs 
associated with collecting data from comparison group members. 


c. Progress to Date and What Will Be Learned in the Future 

FFI program enrollment lagged far below expectations. As of February 2007, one source 
noted that only 35 people had enrolled in FFI (Center for Workers with Disabilities 2007). This 
source and another (GAO 2008b) noted that evaluation of the FFI program was deemed 
infeasible with so few participants and so no formal evaluation has been conducted. 
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6. Mental Health Treatment Study 50 
a. Intervention and Target Population 

In September 2005, SSA awarded a contract to Westat to design, implement, and evaluate a 
demonstration that tests whether better access to mental health treatment and employment 
supports will lead to improved health and functioning, increased employment and earnings, and 
reduced reliance on public benefits among DI beneficiaries with schizophrenia and affective 
disorders. The rationale for the study is that DI beneficiaries with schizophrenia and affective 
disorders represent a large and growing share of all DI beneficiaries, and many have conditions 
that are treatable, but they do not receive adequate treatment or fulfill their employment potential 
due to lack of health insurance coverage and a lack of access to needed behavioral health and 
employment supports. 

The Mental Health Treatment Study (MHTS) is being conducted in 22 sites nationwide. 
Over an 18-month recruitment period, the study plan calls for enrollment of 3,000 DI 
beneficiaries with schizophrenia or affective disorders, ages 18 to 55. Half of the enrollees were 
to be randomly assigned to treatment and half to control. Treatment group members are to 
receive a 24-month intervention comprised of the following components: 


• Health insurance coverage for participants without insurance, and supplemental 
coverage or assistance for those with inadequate insurance 

• Coverage of all out-of-pocket costs associated with all behavioral health services 

• Systematic management of medications 

• Employment supports that are based on the Individual Placement and Support 
supported employment model (Drake 1998; Drake et al. 1999), which includes 
placement in competitive employment, employment supports integrated with 
treatment, and continuous follow-along supports 

A key feature of the intervention is the integrated treatment approach. Under this approach, 
all of a participant’s supported employment and behavioral health services are coordinated and 
provided by a single treatment team located at the local demonstration site. 


b. Evaluation Activities 

The planned MHTS evaluation will include implementation, participation, impact, and cost 
analyses. 


50 Based on information from: SSA (2007); http://www.ssa.gov/disabilityresearch/mentalhealth2.htm; SSA- 
RFP-05-1044 issued July 8. 2005; Frey (2006); and Frey et al. (2008). 
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The implementation analysis will document each site’s adherence to evidence-based 
practices and the study protocol. The participation analysis will assess the beneficiary 
characteristics that predict study enrollment among members of the target population. 

The impact analysis will be based on a randomized control group design and will measure 
the impact of the intervention on a variety of outcomes, particularly those related to employment, 
employment stability, and job quality. The analysis will also examine the impact of the 
intervention on benefit receipt, health care utilization, health and functioning, and quality of life. 
The impact analysis will include an assessment of the degree to which treatment impacts varied 
across treatment group members with different characteristics. 

The cost analysis will document the costs associated with the various types of services and 
medications provided to treatment group members, and assess the relationships between 
participant characteristics and variations in intervention costs. 

The evaluation will rely on SSA administrative data, data collected via a baseline survey of 
treatment and control group members, seven quarterly interviews with treatment group members, 
and a final follow-up survey of treatment and control group members. 


c. Progress to Date and What Will Be Learned in the Future 

MHTS enrollment began in October 2006. If the study proceeds as originally scheduled, the 
findings from the final evaluation should become available sometime in 201 1. Assuming that the 
study is able to meet its sample size targets, the evaluation will be able to produce rigorous 
estimates of the impact of the intervention on functioning, employment, and benefit receipt. The 
MHTS will demonstrate whether the provision of adequate health care and employment supports 
can result in significant changes in health, employment, and benefit receipt outcomes among 
beneficiaries with schizophrenia or affective disorders. From the information available, it is 
unclear to what extent the evaluation will consider impacts of the intervention on other public 
programs, or consider more generally the social costs and benefits of the intervention. 


7. Substantial Gainful Activity Level Increase 
a. Intervention and Target Population 

On July 1, 1999, the definition of the substantial gainful activity (SGA) level was raised 
from $500 to $700 of earned income per month. The SGA level is the amount of income that an 
individual with a disability applying to the DI and SSI program may earn to be initially 
considered eligible for the programs. It is also the amount that DI beneficiaries may earn without 
losing disability benefits. The SGA level provides a substantial incentive for DI beneficiaries to 
work only up the income level of the threshold itself to avoid the complete loss of benefits. 51 The 
adjustment in 1999 was the first of its kind since 1990, and reflected the growth in average 


51 SSI recipients lose benefits gradually as earnings rise, whereas DI beneficiaries lose all benefits after 
earnings have exceeded SGA for a specified period of time. 
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wages since that time. In December 2000, SSA finalized a rule that allows for the annual 
indexing of the SGA level. Adjustments to SGA are now applied annually to reflect annual 
growth in national average wages. The nonblind SGA level in 2008 was $940 per month. 

There are at least two potential effects of this change, depending on the employment status 
of an individual before the change went into effect: 


• For beneficiaries earning below the former SGA level of $500, the change represents 
an opportunity to increase earnings without loss of benefits. Those already working 
might increase the number of hours they work to take advantage of this opportunity, 
and those who did not perceive the effort of finding a job worthwhile under the old 
limit might decide to seek employment under the new limit. 

• For nonbeneficiaries earning more than the former $500 threshold but at or near the 
current $940 level, there is an incentive to restrict earnings to the new threshold to 
become eligible for disability benefits, which might result in fewer program exits and 
a decrease in earnings among members of this group. 


The presumption is that the first effect will dramatically outweigh the second — that is, there 
are far more individuals with disabilities working just below the threshold than there are working 
just above it. It is anticipated that this change will facilitate meaningful work among 
beneficiaries by reducing the likelihood that their return-to-work efforts will result in the 
immediate loss of cash benefits. 


b. Evaluation Activities 

In 2002, GAO released a report that assessed the effects of the SGA level on the DI program 
(GAO 2002). The assessment by GAO was mandated under the Ticket Act. GAO’s study 
considered the effects of the SGA level on the work patterns of DI beneficiaries as well as the 
effects of the SGA level on DI program entry and exit rates. The study was based on a review of 
the economic and disability literature related to the effects of the SGA; an analysis of DI 
program data covering the period of 1985 through 1997; and interviews with SSA policy 
officials, academic experts, and representatives from disability advocacy groups. 

GAO’s quantitative analysis relied on data from SSA’s Continuous Work History Sample 
(CWHS). The CWHS is a file representing a longitudinal sample of one percent of all active 
Social Security accounts. The file contains data on earnings for purposes of analyzing the 
lifetime earnings patterns of individuals. The earnings data are annual, however, so monthly 
SGA levels had to be annualized, with the analysis based entirely on the annualized SGA 
amounts. The CWHS also contains data abstracted from SSA administrative files, including 
information on DI program eligibility and demographic characteristics. From the CWHS file, 
GAO selected a subsample of approximately 90,000 DI beneficiaries eligible for DI benefits at 
some time between 1984 and 1998. During the study period, the SGA level for nonblind 
individuals increased from $300 per month (from 1980 to 1989) to $500 per month (from 1990 
to mid- 1999). The SGA level for blind individuals increased steadily over the entire period, from 
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$580 per month in 1984, to $1,050 per month in 1998. GAO also drew a subsample of about 
10,000 DI beneficiaries who reached age 65 during the 1987 to 1993 period to examine 
beneficiary work activity after conversion to the retirement program, where SGA earnings 
restrictions do not apply. 

GAO analyzed the annual earnings data to determine if trends observed over the 1984 to 
1998 period were suggestive of any effects of the SGA level on the earnings of DI beneficiaries. 
If the SGA level does indeed affect earnings, a priori, one would expect to see evidence of the 
following as a consequence of the monthly nonblind SGA level increase from $300 to $500 in 
1990: 

• A greater share of DI beneficiaries working during the post- 1990 period 

• Among working DI beneficiaries, higher levels of earnings in the post- 1990 period 

• An increase in the share of working beneficiaries with earnings between $300 and 
$500 per month during the post- 1990 period 


One might also expect to see blind beneficiaries generally working at higher rates and levels 
than nonblind beneficiaries, as in each year, they were subject to substantially higher SGA limits. 
In addition, one might expect to find relatively higher levels of employment after conversion to 
the retirement program, when a beneficiary is no longer subject to the SGA earnings limits. 

With respect to effects of the SGA on program entry and exits, an increase in the SGA level 
is hypothesized to increase program entry because those working between the old and new SGA 
limit would now qualify for benefits, and those working just above the new limit might restrict 
their earnings to qualify for the program. With respect to program exit, the hypothetical effect of 
an increase in the SGA level is ambiguous. On the one hand, beneficiaries can earn more and 
remain eligible for DI, and thus fewer may leave the program due to earnings. On the other hand, 
allowing beneficiaries to engage in greater work activity could, in the long run, result in more 
beneficiaries becoming capable of earning at levels leading eventually to exit from the disability 
rolls. 


c. Findings 

In summary, GAO found only limited evidence of an effect of SGA on earnings and no 
conclusive evidence of any impact on program entry or exit. 

GAO concluded that the SGA level affects the work patterns of only a small proportion of 
DI beneficiaries. It found that, on average, only about 7 percent of DI beneficiaries who worked 
in any given year during the study period had earnings near the annualized SGA level (between 
75 and 100 percent of SGA). These beneficiaries comprised only about 1 percent of all DI 
beneficiaries. Even among beneficiaries who had earnings near the SGA level in any given year, 
most experienced a substantial decline in earnings over time. For example, almost half of those 
with earnings near the SGA level in 1985 had no earnings by 1989. GAO did find some evidence 
that the SGA might affect the earnings of some beneficiaries. About 13 percent of beneficiaries 
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with earnings near the SGA level in 1985 still had earnings near the SGA level in 1995, even 
though the level was increased during that period. Among beneficiaries converting to the 
retirement program, GAO found that, among those with no earnings during the three years prior 
to conversion, about 7 percent had earnings in one or more years following conversion (between 
ages 66 and 68). The rate of employment for this group was more than double the rate of a 
similarly defined comparison group of DI beneficiaries ages 58-60 (3 percent had earnings in one 
or more years), suggesting that the SGA has an effect on earnings. Although trends in program 
entry and exit were analyzed, the study could draw no conclusions from the findings regarding 
the effect of the SGA on entry or exit given significant data limitations and a variety of factors 
unrelated to the SGA level that likely influenced the aggregate trends observed. 

The GAO study is flawed in many respects, primarily due to data limitations. The data 
available only permitted analyses of annual, rather than monthly, earnings. This makes it 
impossible to precisely identify any effects of the SGA on earnings because the earnings are 
averaged over a 12-month period. The averages may or may not correspond to the actual 
monthly earnings of beneficiaries and will be particularly poor indicators of monthly earnings 
among beneficiaries who work sporadically. The data also could not distinguish beneficiaries 
who (1) had not completed their trial work period, during which time any level of earnings is 
permitted); (2) had completed their trial work period and had not reach the 36th month of the 
Extended Period of Eligibility, when, after a three-month grace period, earnings above SGA in 
any month result in suspension of benefits for that month only; and (3) had reached or gone past 
the 36th month, when earnings above SGA in any month result in termination of DI benefits. 
Thus, the analysis includes a mix of earnings observations from among beneficiaries who (1) had 
no incentive to keep earnings below SGA; (2) had an incentive to keep earnings below SGA, to 
avoid temporary benefit loss; and (3) had an even stronger incentive to keep earnings below 
SGA, to avoid benefit termination. The data used in the study also could not distinguish between 
blind and nonblind beneficiaries. Therefore, the analysis includes a mix of beneficiaries subject 
to different SGA levels. In addition, comparisons that might have provided further evidence of 
the effect of the SGA on earnings could not be undertaken — for example, comparisons between 
the two groups subject to very different SGA levels and comparisons over time of blind 
individuals subject to a steadily increasing SGA. Finally, the data were rather old. It is unclear 
whether the findings from the 1984 to 1998 period would still be applicable today given the 
considerable changes around employment issues that have happened with the Social Security 
disability programs since the passage of the Ticket Act in 1999. The GAO did not conduct an 
evaluation of the 1999 SGA increase. 

Despite the data limitations, there are other analyses the GAO study could have undertaken 
that might have provided additional, or more compelling, evidence of the effect of the SGA level 
on earnings. GAO chose to focus much of its analysis on the number and share of beneficiaries 
(and working beneficiaries) with earnings far below, near, and above SGA. While interesting, 
such an analysis provides little information about the effect of SGA on earnings and makes little 
use of the natural experiment provided by the 1990 SGA increase. It would have been interesting 


52 The trial work period allows beneficiaries to earn any amount without losing benefits for a period of nine 
months within a 60-month period. The Extended Period of Eligibility starts in the first month after the trial work 
period ends. 
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to see the data presented in a manner that allowed one to examine the percent of beneficiaries 
(and working beneficiaries) earning at the old and new SGA levels (absolute dollar levels) both 
before and after the policy change. Similarly, the report presents a table of the earnings 
distributions in 1985 and 1997. The table shows the number of beneficiaries at various annual 
earnings levels after adjustment for inflation. The report concludes that “even with the 67 
percent increase in the SGA level in 1990, the earnings distribution of DI beneficiaries did not 
change considerably from 1985 to 1997” (p. 12). This is hardly evidence of the lack of an effect 
of the SGA on earnings given that, after adjustment for inflation , the 67 percent nominal increase 
in SGA (from $300 in 1985 to $500 in 1997) amounts to a five percent decrease in inflation- 

co 

adjusted dollars. It would have been more interesting if the GAO report had included a percent 
distribution of beneficiaries by both inflation- adjusted and nominal dollars 

The study emphasizes the relatively small share of beneficiaries who work, and even smaller 
share who work near SGA. The conclusion from these observations that the SGA level affects 
very few beneficiaries seems to ignore the obvious: if a program places an extreme restriction on 
earnings to maintain eligibility, very few participants are likely to bother attempting work. 
Perhaps the most persuasive evidence of the effect of the SGA level on earnings was GAO’s 
findings with respect to beneficiary work activity after conversion to the retirement program. To 
find that employment activity among this group was more than double that of a comparison 
group of older DI beneficiaries is quite compelling. If post-retirement age individuals with 
disabilities are responsive to the elimination of the SGA restriction, one can speculate that the 
response among younger beneficiaries who have significantly more years of potential earnings 
ahead of them might be considerable. 


8. Ticket to Work Program 
a. Intervention and Target Population 

The Ticket to Work and Work Incentives Improvement Act of 1999 established the Ticket to 
Work program (TTW). TTW is a national program that provides eligible DI and SSI disability 
beneficiaries with a Ticket, which can be used to obtain vocational rehabilitation (VR) or 
employment services through a participating provider, called an Employment Network (EN). 
ENs can be any type of entity willing and able to provide employment-related services to 
beneficiaries. There are very few restrictions on the types of entities that can become ENs. SSA, 
through a contractor, is responsible for recruiting a large network of ENs that can accommodate 
beneficiary demand for services in all states and territories. 

Under TTW, a beneficiary can assign a Ticket to any EN willing to accept it. ENs are paid 
by SSA only if the beneficiary goes to work and meets or exceeds specific employment and 
earnings targets, and provided that the EN supplies sufficient evidence of the beneficiary’s 
earnings. The earnings targets and payments differ depending on in which of two TTW payment 
systems the EN chooses to participate (milestone-outcomes or outcomes only). Under both 


53 Assuming an average annual rate of wage growth of five percent from 1985 to 1997, $500 in 1997 would be 
equivalent to about $280 in 1985 constant dollars. 
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systems, however, most (or all) payments are not made to ENs unless a beneficiary is working 
and earning enough to reduce SSI and/or DI benefits to zero for 60 months. State VR agencies 
may choose to participate in either of the two TTW reimbursement schemes, or to receive 
payment based on the traditional system by which SSA has for many years reimbursed state VR 
agencies for services provided to beneficiaries if the beneficiary earns more than SGA for at least 
nine months. There is no benefit reduction requirement under the traditional system. 

The underlying rationale for the program is that some beneficiaries currently lack the 
resources necessary to return to work at a level above the SGA level, either because they do not 
have easy access to such services or because they and their providers lack the incentive to invest 
resources in return-to-work activities that will lead to SS 1/D 1 program exit. TTW confers upon a 
beneficiary a means to access those resources in a less restrictive manner than under SSA’s 
traditional VR payment program. TTW was expected to increase beneficiary demand for 
employment-related services and activities, to increase the number and diversity of providers 
serving SSI and DI beneficiaries, and to create incentives for providers to help beneficiaries 
achieve a level of earnings that will result in exit from the disability rolls, and thereby reduce 
Social Security disability program expenditures. 

Individuals ages 18 to 64 who are currently receiving either DI or SSI disability benefits are 
eligible to participate in TTW. Such individuals may have either permanent impairments 
(impairments for which medical improvement is not expected) or temporary impairments 
expected to last at least 12 months (impairments where medical improvement is expected). 
Originally, to be eligible for TTW, those individuals with temporary impairments must have first 
undergone a CDR, and have been found to have experienced insufficient medical improvement 
to allow a return to work. In May 2008, the regulations were amended to allow those with 
medical improvement expected who had not yet undergone the first CDR to receive Tickets and 
be eligible to participate in TTW. 

TTW was implemented in three phases: between February and June 2002, TTW was 
implemented in 13 states; between November 2002 and September 2003, TTW was implemented 
in an additional 20 states and the District of Columbia; between November 2003 and September 
2004, TTW was implemented in the remaining 17 states and U.S. territories. 


b. Evaluation Activities 

In June 2003, SSA contracted with MPR to conduct a five-year evaluation of TTW, based 
on an evaluation design that had been previously developed by The Lewin Group. 54 The 
evaluation is intended to address five broad issues: 


• Does TTW improve the supply of employment-related services to beneficiaries? 

• Does TTW increase beneficiary use of employment services? 


54 See Stapleton and Livermore (2002). 
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• Are TTW services effective in improving employment outcomes? 

• Does TTW generate net disability program savings? 

• Are there other social consequences associated with TTW? 


The Evaluation has Three Primary Components: 

The process evaluation uses survey, administrative, and qualitative data to document how 
the program was implemented; assess the effect of the program on the market for employment- 
related services (beneficiary demand, provider supply, dynamics, funding sources); and provide 
contextual information to help interpret impact analysis findings. 

The participation analysis uses administrative and survey data to assess who participates in 
TTW and who does not; the relationships between participation and both beneficiary and area 
characteristics; and reasons for nonparticipation. 

The impact analysis uses administrative data to produce estimates of impacts on enrollment 
in services, employment, annual earnings, exits from the disability programs for work, and net 
disability program savings. The impact estimation relies on a quasi-experimental design, where 
the outcomes of pre-implementation states are compared with the outcomes of post- 
implementation states, taking advantage of the phased implementation of the program. 

Additional outcome analyses will supplement the impact analyses by documenting outcomes 
for which it is not possible to produce rigorous impact analyses because the data required for 
impact estimates are not available. These outcomes include the nature of employment services 
received by TTW participants and by other beneficiaries, the cost of services provided by ENs, 
beneficiary experiences with use of TTW and satisfaction with their experience, beneficiary 
income, beneficiary use of other programs, and impacts on other programs. The beneficiary 
outcomes are measured via the survey, and other outcomes are measured from various sources. 


c. Findings 

To date, four extensive TTW evaluation reports have been released: a preliminary process 
evaluation report developed by The Lewin Group (Livermore et al. 2003) and three evaluation 
reports developed by MPR (Thornton et al. 2004, 2006, and 2007). The major findings from the 
most recent of these reports include the following: 


• From SSA’s perspective, TTW was challenging to implement for several reasons: the 
short timeframe; limited resources available to implement the Ticket Act provisions; 
administrative and system inadequacies requiring substantial effort to address; and the 
considerable effort involved in EN recruitment. 

• Beneficiary participation in the program has been limited but continues to grow. As 
of December 2004, 1.4 percent of all TTW-eligible beneficiaries in Phase 1 states had 
assigned a Ticket to a provider. 
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• Beneficiary survey data suggest some reasons for the limited beneficiary participation 
in TTW: lack of awareness of the program and lack of interest in work among 
beneficiaries. Only about one-third of all beneficiaries had heard of the program at the 
time they were interviewed in 2004. Among all beneficiaries, only 30 percent indicate 
having goals that include work or career advancement, and only 26 percent see 
themselves working within the next five years. 

• Although participation in TTW has been low, the survey data suggest that there is 
potential demand for the program. About 13 percent of all beneficiaries worked at 
some time during the previous year, and a substantial share of all beneficiaries have 
goals that include work or career advancement and see themselves working within the 
next five years. However, only 15 percent of all beneficiaries and 52 percent of TTW 
participants see themselves working and earning enough to leave the disability rolls 
within the next five years. Thus, while potential demand for the program might be 
substantial, the ability to succeed in the program might be limited, given the 
program’s implicit definition of success — work at levels sufficient to reduce disability 
benefits to zero. 

• Most Tickets (over 90 percent) have been assigned to state VR agencies and the vast 
majority of these have been assigned under the traditional reimbursement method 
over one of the two new TTW reimbursement schemes. This suggests that TTW has 
had little impact on the delivery of services to beneficiaries, as state VR agencies 
have historically been the principal source of VR and employment-related services 
for beneficiaries. 

• While a large number of providers have enrolled in the program as ENs 
(approximately 1,300), few are accepting any or many Tickets. Only about 40 percent 
of all ENs have taken any Ticket assignments, and only about 20 percent have 
accepted more than five Tickets. Reasons for the low levels of participation among 
ENs include lack of up-front funding within the EN payment systems; the high cost of 
screening beneficiaries; administrative burden and complexity of the program; 
discomfort with the low level of risk-sharing within the EN payment systems; a belief 
that the value of EN payments is low relative to payments from other sources; 
concerns about the ability to mingle Ticket payments with other sources of funding; a 
belief that few potential clients will work at levels sufficient to generate payments; 
and lack of capacity to serve significant numbers of Ticket holders. 

• TTW appears to have had an impact on the number of beneficiaries enrolled for 
employment services but has had no measurable impact on earnings or benefit 
receipt. The lack of measurable impacts on earnings and benefits might reflect the 
fact that the quasi-experimental evaluation design, which relied on comparisons of 
outcome changes for early and late implementation states, cannot detect effects that, 
although important, are small relative to historical state-level variation in earnings 
and benefits. Even if the TTW was successfully leading to increases in earnings and 
exits from DI, effects would likely be small in the first two years, as participants find 
work, complete their trial work periods, and eventually experience earnings 
reductions. Unfortunately, the quasi-experimental method cannot be extended beyond 
the first two years, because TTW was in place in all states after two years. 
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In July 2008, SSA implemented very substantial revisions to TTW regulations. The changes 
are expected to increase the incentives that providers have to serve beneficiaries and might 
increase beneficiary interest in earning their way off the rolls. SSA expects to continue the TTW 
evaluation. Although the ability to conduct an impact evaluation is limited by the lack of a 
suitable comparison group, it will not be difficult to detect a marked increase in the number of 
beneficiaries who exit the rolls because of work, as historically so few have done so. While such 
an increase would not necessarily be entirely attributed to TTW, it might be an indicator of 
impact. 


H. SUBSTANCE ABUSE AND MENTAL HEALTH SERVICES ADMINISTRATION 

I. Employment Intervention Demonstration Program 
a. Intervention and Target Population 

The Employment Intervention Demonstration Program (EIDP) was a multisite randomized 
controlled trial of the effectiveness of supported employment for people with psychiatric 
disabilities in eight locations across the U.S. SE programs use a rapid job search approach to help 
clients obtain jobs directly (rather than providing lengthy assessment, training, and counseling) 
and provides them with ongoing support to maintain and improve their earnings after they start 
work. The study was funded by the Center for Mental Health Services of the Substance Abuse 
and Mental Health Services Administration. The purpose of the study was to generate knowledge 
about effective approaches for enhancing employment among adults with severe mental illnesses 
(Cook et al. 2002). Half of all participants had a schizophrenia spectrum diagnosis and another 
40 percent were diagnosed with major depression or bipolar disorder. Over half had a secondary 
diagnosis of substance abuse. 

The experimental study group received services under different supported employment 
service models designed specifically for people with psychiatric disabilities. All of the 
experimental conditions had five characteristics: (1) integrated services delivered by a 
multidisciplinary team that met three or more times per week to plan and coordinate employment 
interventions with case management and psychiatric treatment; (2) placement into competitive 
employment, defined as jobs paying at least minimum wage, in regular, socially integrated 
community settings; (3) development of jobs tailored to personal career preferences; (4) use of a 
job-search process beginning immediately after program entry and moving as quickly as the 
individual desired; and (5) provision of ongoing vocational supports freely available throughout 
the entire study period. 

The control groups received services as usual (that is, whatever was typically available in 
the participants’ local communities), unenhanced versions of the experimental models, or 
Clubhouse services. Generally, individuals in the control group received lower amounts of 
vocational services although they received equivalent amounts of psychiatric services in 
comparison to experimental group participants. 
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b. Evaluation Activities 


The study randomly assigned over 1,600 participants to experimental and control groups at 
the eight EIDP study sites, and the researchers followed them for two years. The study 
documented vocational outcomes, including competitive employment, earnings, employment 
status, benefit receipt, and number of hours worked. 


c. Findings 

The EIDP evaluation findings have been reported in a number of published papers (Burke - 
Miller et al. 2006; Cook et al. 2005a, 2005b, 2006; Leff et al. 2005; and Razzano et al. 2005). 
Some of the key general findings of the study include the following (Cook 2007): 


• Experimental group participants were more likely than the control group participants 
to achieve competitive employment, work more than 40 hours in a given month, and 
earn more money. People with severe mental illness who received well-integrated and 
coordinated vocational and clinical services had significantly better employment 
outcomes than those who received noninte grated services. 

• Integrated employment services resulted in positive employment outcomes regardless 
of consumers’ personal characteristics, diagnoses, work histories, receipt of SSA 
disability income, and functioning levels. 

• The more vocational services participants received, the better the employment 
outcomes. 

• Personal characteristics such as type and intensity of mental disorders and psychiatric 
symptoms influenced employment outcomes within vocational programs. 

• Employment outcomes for all study subjects were negatively related to the level of 
unemployment in the study area. Impacts were largest in the study areas with the 
lowest unemployment, but were substantial even in areas with high unemployment. 


The study clearly demonstrated positive impacts on employment of supported employment 
approaches that integrate health and vocational supports for persons with severe mental illness. 
In addition, the advantage of supported employment over other programs increased over the 24- 
month study period, making it apparent that programs offering ongoing support and services that 
build on career achievements had greater success. The findings support the importance of 
providing ongoing supported employment services as a best practice in vocational rehabilitation 
for people with psychiatric disabilities. 
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